Yueyang Chen

wwwwww

• 03/22 - present, ETL Pipeline Generator, Nike Inc, Shenzhen, China, Design and implemented a config-based ETL pipeline generator that serves to reduce ETL maintenance effort and improve model explainability for bot detection domain experts., Achieved separation of business logics from ETL implementation, which has reduced 90% maintained code., Developed a devops automation tool that transpiles and deploys the code to Azure Databricks within 3 seconds., Python, PySpark, Snowflake, Hive, Airflow, Docker, Event Driven Design
• 10/20 - 10/20, Data Collaboration Framework, Boston Consulting Group, New York, NY, Individually designed and implemented a multi-cloud-provider, abstract framework for developing complex ETL, data quality testing, and data analysis, which resolved the critical issue of engineering collaboration with the client team., The framework runs on top of data lake and data warehouse., Azure Databricks, ADLS, Pyspark, Python, Domain Driven Design (DDD), Event Driven Design, Docker
• 04/20 - 07/20, EMR-based ETL Framework, Boston Consulting Group, New York, NY, Implemented several features to enhance an Spark-based ETL Framework used by the whole DE department., Tackled the Spark-Redshift slow-writing issue, which further enabled the DE department to work with Redshift., Presto, Snowflake, MySQL, SQL Server, PostgreSQL, AWS EMR, Redshift, CloudWatch, CloudFormation, Dynamo, RDS, S3, Python, Terraform, PySpark, Docker
• 10/19 - 12/19, Data Architecture Migration, Boston Consulting Group, New York, NY, Took lead in migrating 30+ individual AWS Glue scripts to modularized, organized PySpark code base, which enabled the client on a major market campaign based on customer identity data merged from 3 data systems., Refactoring the code and completing the migration had reduced $26k AWS costs each year for the client., Presto, SQL, AWS EMR, S3, Redshift, Glue, Athena, Dynamo, RDS, CloudWatch, Python, Docker
• 2020 - 2020, CRM System Enhancement, Boston Consulting Group, New York, NY, Expanded the capability of a product-based CRM system to be able to sell business services., Conduct conversation directly with the non-technical client team to discuss technical requirements., SQL Server, Javascript, Docker
• 10/17 - 08/18, Framework for developing distributed data pipelines, Myers Media Group LLC, San Diego, CA, Took lead in implementing an event-driven, abstract framework for developing distributed data crawlers and ETL pipelines, which runs on data lake and data warehouse., Achieved 99% prevention of data duplication., Python, Django, PostgreSQL, AWS SQS, ElasticBeanstalk, ECS, Athena, Glue, SNS, Javascript, Event Driven Design, Docker