The data team, composed of 7 members who work closely with both the DevOps and the DataScience teams, is in charge of developing and monitoring the data collect pipeline.
The collect, which processes a few terabytes of data per day, has been deployed on a Google Cloud Platform environment and is critical to AB Tasty.
Different GCP environments are available to ensure the good development and deployement of features as well as data modelisation and documentation.
What you will do :
Create or update features of the data processing pipelines using GCP components as dataflow, bigquery, bigtable
Develop new services, dashboard to ensure the data quality in real-time and design a moniting process
Participate in the design phases and implementation of new micro-services and machine-learning features plugged into the pipeline
Explore and implement new data sources
Monitor the pipeline
What we're looking for :
Strong knowledge of at least one of the following technologies : Apache Beam ou Spark, et Hadoop
Knowledge in Java, Python, and / or Golang
A least 2 years of experiences as a (Big) Data Engineer
Experience with cloud platform is a plus
Good level of English (written and spoken)