- Minimum 2 years of experience building and deploying data pipelines
- Hands-on experience with at least one cloud-based data warehouse e.g., Snowflake, BigQuery; experience with big data formats e.g., Delta Lake, Parquet, Avro would be an asset
- Good knowledge of relational and dimensional data models; able to interpret and understand physical data models and apply data rules and constraints as required to create data pipelines; prior data warehousing architecture knowledge would be an asset
- Hands-on experience building ETL/ELT data pipelines via custom-coded scripts (e.g., Spark, Python, JAVA, SQL stored procedures) OR via integration platforms (e.g., PowerCenter, DataStage, Talend); by following various standards and best practices such as coding and naming standards, version control, code promotion, testing, and deployment
- Strong verbal and written communication skills as well as excellent collaboration skills are required to participate and engage in highly technical discussions regarding data solutions
- Demonstrated ability to self-learn and master new data tools, platforms, and technologies within a short ramp-up period under conditions of limited formal training and coaching
- Experience with data orchestration in Apache Airflow, Cron, or other schedulers is a strong asset
- Knowledge of microservices architecture and container technology such as Kubernetes and Docker would be a definite asset
- Experience managing data platforms via infrastructure-as-code eg: Terraform would be a strong asset
Base City:
Remote-Canada - Remote
Salary:
No Salary therefore no Star!
Rating:
Self-taught:
Position Type:
Full-time
Position Keywords:
- Airflow
- Apache
- APIs
- Avro
- Big Query
- Cron
- Data Ingestion
- Data Lake
- Data Lake
- data model
- Data Pipelines
- Data Stage
- Data Warehouse
- Delta Lake
- Docker
- ETL
- Infrastructure as Code (IaC)
- Java
- Kubernetes
- Metadata
- NoSQL
- Parquet
- Physical Data Models
- Power Center
- Python
- RDBMS
- Snowflake
- Spark
- SQL
- StackAdapt
- Talend
- Terraform
Experience:
2 Years Data Pipelines
Other Experience:
About the Job:
StackAdapt is a self-serve advertising platform that specializes in multi-channel solutions including native, display, video, connected TV, audio, in-game, and digital out-of-home ads. We empower hundreds of digitally-focused companies to deliver outcomes and exceptional campaign performance everyday. StackAdapt was founded with a vision to be more than an advertising platform, it’s a hub of innovation, imagination and creativity.
We have an exciting opportunity in the newly formed Enterprise Data Office (EDO) with its mandate to serve the business leaders and data stakeholders at StackAdapt with trusted official reporting and governed self-service analytics. Reporting to the Director of Enterprise Data Office, the ETL Developer will be responsible for end-to-end data pipeline development from ingesting the raw data into the centralized data lake, to transforming the data into business-friendly data models on the Enterprise Data Warehouse that can be easily consumed by the business through Business Intelligence applications and other downstream processes.
What they want you to do:
This role will assess the data requirements provided by the Manager of Business Data Analysis and collaborate with other team members and the source data owners to design and build automated batch or near-real-time data ingestion pipelines that will ingress the necessary data from the source, which may be a variety of formats and mediums such as relational/non-relationship databases, flat files, and applications, into the centralized data lake. Subsequently, under the guidance of the Data Architect, who will provide the target data models, the ETL developer will build the transformation pipelines necessary to materialize various types of data models into the Enterprise Data Warehouse. The ETL developer will work collaboratively with the Data Architect in the deployment, automation, and orchestration of these newly developed data pipelines, and collaborate with the BI engineers to ensure the smooth connection of these data models into the BI semantic layer so that it can be leveraged easily by business users. Finally, the ETL developer will also assist the Data Architect and the Manager of Business Data Analysis in the maintenance and support of the data pipeline operations as well as the overall EDO environments.
StackAdapt is a Remote-First company. We are open to candidates located anywhere in Canada for this position.
What you'll be doing:
- Build reliable data ingestion pipelines to extract data from a variety of data sources including databases (e.g., RDBMS/NOSQL/file stores), applications (via API), flat files, etc into the Data Lake with appropriate metadata tagging
- Build data transformation pipelines to transform the raw data and materialize the data models designed by the Data Architect into the Enterprise Data Warehouse
- Deploy developed pipelines into production in adherence with deployment best practices to ensure a seamless rollout
- Orchestrate data pipelines via batch, near-real-time, or real-time operations depending on requirements to ensure a seamless and predictable execution
- Support the day-to-day operation of the EDO pipelines as well as the EDO environment by monitoring alerts and investigating, troubleshooting, and remediating production issues
- Work with members of the Enterprise Data Office to ensure the stability and optimization of the data pipelines to meet the required SLA