You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
<bclass="italic-text">Company's [<ahref="https://github.com/datajoint">Open-source Github</a>] and [<ahref="https://datajoint.com/works">Commercialized Product</a>]
140
+
</div>
141
+
<div>
142
+
<bclass="italic-text">* AWS:</b> Administrated the company's AWS account and several other customers' AWS accounts. Configured <b>VPC</b>, <b>Subnet</b>, <b>Security Groups</b>, <b>IAM</b> role and policies, <b>S3</b> lifecycle management, <b>EFS</b> access point, <b>EC2</b> instances, <b>RDS</b> instances, <b>Lambda</b> triggered by <b>SQS</b> or <b>EventBridge</b>, <b>SNS</b> and <b>SES</b>, <b>CloudWatch</b> metrics and alarms, <b>Route 53</b> DNS records, <b>Secrets Manager</b> for deployment secrets.
143
+
</div>
144
+
<div>
145
+
<bclass="italic-text">* CI/CD: </b> Developed generic <b>Github Actions</b> reusable workflows used by <b>30+</b> repositories followed by <ahref="https://www.conventionalcommits.org/en/v1.0.0/">Conventional Commits</a>, <ahref="https://learn.microsoft.com/en-us/devops/develop/how-microsoft-develops-devops">Release Flow</a> and <ahref="https://opengitops.dev/">GitOps</a> best practices, to automate build, test, release, publish private or open-source <b>Python</b> packages[<ahref="https://pypi.org/search/?q=datajoint">PyPI</a>] or deploy <b>Docker</b> images[<ahref="https://hub.docker.com/u/datajoint">Dockerhub</a>].
146
+
</div>
147
+
<div>
148
+
<bclass="italic-text">* Kubernetes: </b> Provisioned Kubernetes clusters for development, staging and production environments using <b>k3d or kOps</b>. Developed utility <b>bash</b> scripts with <b>helm</b> and <b>kubectl</b> to manage Kubernetes clusters more efficiently, including configuring <b>Nginx ingress</b> controller, cert manager with <b>Let's encrypt</b> issuer, <b>Cillium</b> Container Network Interface(CNI), IAM Roles for Service Account(<b>IRSA</b>), <b>Cluster Autoscaler</b>, AWS Elastic Load Balancer(<b>ELB</b>) or deploying applications like Percona XtraDB Clusters, Keycloak, JupyterHub, Flask and ReactJS based web application, etc.
149
+
</div>
150
+
<div>
151
+
<bclass="italic-text">* Ephemeral Worker Clusters: </b> Designed and developed a worker lifecycle manager using Python in about a month to fulfill an <b>urgent</b> business requirement. This development <b>polls</b> jobs from a MySQL database, then provisions and configures ephemeral EC2 instances by <b>Packer(pre-build AMI), Terraform and cloud-init</b> to compute jobs <b>at scale</b>; implemented AWS S3 mount to significantly reduce raw data downloading <b>overhead</b> and added EFS as a file cache for intermediate steps to improve computation <b>failover</b>; configured <b>NVIDIA CUDA toolkit</b> and <b>NVIDIA container runtime</b> for <b>GPU</b> workers.
152
+
</div>
153
+
<div>
154
+
<bclass="italic-text">* Platform Automation: </b> To provision or terminate AWS resources using <b>boto3</b> or <b>Terraform</b>; manage customers' <b>RBAC</b> permissions using Keycloak and Github REST API; generating usage and billing report with <b>AWS S3 Inventory</b> report, <b>AWS CloudTrail</b> and <b>AWS Cost and Usage</b> report, made a <b>Plotly Dash</b> to analyze cost and usage efficiency.
155
+
</div>
156
+
<div>
157
+
<bclass="italic-text">* Jupyterhub: </b> Configured and maintained Jupyterhub deployment on a Kubernetes cluster with <b>Node Affinity</b> to assign pods onto different nodes by requirements and <b>Cluster Autoscaler</b> along with <b>AWS Auto Scaling Group</b> to accommodate <b>100+</b> active users; improved base images' <b>build time</b> and maintenance <b>overhead</b>.
158
+
</div>
159
+
<div>
160
+
<bclass="italic-text">* Observability:</b> Implemented small part of the metrics and alerts using <b>AWS CloudWatch</b>, and then later integrated <b>Datadog</b> for Kubernetes clusters' and ephemeral EC2 instances' metrics and logging through <b>OpenTelemetry</b> protocol, synthetic API testing, UI/UX monitoring.
161
+
</div>
162
+
<div>
163
+
<bclass="italic-text">* Security:</b> Set up codebase <b>vulnerability</b> scan with FOSSA; Set up <b>AWS Secrets Manager</b> working with <b>External Secret Store Operator</b> to secure Kubernetes secrets; Deployed and administrated self-hosted <b>Keycloak</b> for <b>RABC</b> authentication, further integrate it with <b>AWS IAM</b> as an <b>identity provider</b> to access AWS resources through <b>STS</b>, learned about OpenID Connect(<b>OIDC</b>) authentication flow such as authorization code flow, client credential flow, password grant flow etc.
164
+
</div>
165
+
<div>
166
+
<bclass="italic-text">* MySQL Database:</b> Maintained a self-hosted <b>Percona XtraDB Clusters</b> on database <b>daily backup</b> stored on <b>S3</b>, <b>mysqldump</b> backup redundancy, Point-in-Time Recovery(<b>PITR</b>), <b>deadlock</b> detection, slow query log.
<bclass="italic-text">* Interactive Drilling Dashboard: </b>This is an <b>enterprise</b> product that I worked with two more engineers. Developed a <b>Plotly Dash</b> dashboard that visualizes processed data using Bootstrap, CSS media query, <b>Redis</b> and sqlalchemy. Also, implemented a <b>socket</b> service will notify when <b>Airflow</b> pipeline finished processing in order to <b>synchronize</b>(refresh) the dashboard's data.
179
+
</div>
180
+
<div>
181
+
<bclass="italic-text">* CI/CD Pipeline: </b>Set up several <b>Azure Pipelines</b> for continuous development, testing and continuous deployment in <b>dev, test and prod</b> stages. Additionally, made a <b>Jenkins</b> pipeline to work with on-premise infrastructures.
182
+
</div>
183
+
<div>
184
+
<bclass="italic-text">* ML Pipeline: </b>Set up a <b>MLflow</b> server for machine learning experiment logging, parameter tuning, continuous training, model management and model serving.
185
+
</div>
186
+
<div>
187
+
<bclass="italic-text">* ETL Pipeline: </b>Working with a data engineer, set up an <b>Airflow</b> server for our data ETL pipeline.
188
+
</div>
189
+
<div>
190
+
<bclass="italic-text">* Prediction Task Manager: </b>Working with a front-end developer, designed and developed a <b>production</b> web application that supports job queuing and parallel processing for drilling speed prediction using JavaScript, <b>flask</b>, sqlalchemy, <b>celery</b>, RabbitMQ, gunicorn, Nginx, supervisord, Docker and AWS EC2, AWS Cognito Authentication, HTTPS
191
+
</div>
192
+
<div>
193
+
<bclass="italic-text">* Drilling Status Detection: </b>Working with a domain expert, developed two <b>classification</b> models for detecting drilling status using Logistic Regression and Random Forest with the convenience of the MLflow server
194
+
</div>
195
+
<div>
196
+
<bclass="italic-text">* Drilling Speed Prediction: </b>Working with a domain expert, applied Gaussian Process <b>Regression</b> for feature synthesis based on geographical information as well as <b>feature engineering</b> based on correlation matrix and F1 score ranking, built a non-linear regression model using LSTM RNN.
197
+
</div>
198
+
</div>
199
+
</li>
200
+
<li>
201
+
<span>May 2019</span>
202
+
<divclass="content">
203
+
<h3>Graduated Master of Computer Science - Southern Methodist University</h3>
<bclass="italic-text">* Gains: Git Version control, Android updated version features, more customized widgets
221
+
</div>
222
+
<div>
223
+
<bclass="italic-text">* Self assessment: it's good to work remotely in a professional team included designers and developers, however, I still need more understanding in an entire software development lifecycle, not just coding.
224
+
</div>
225
+
</div>
226
+
</li>
227
+
<li>
228
+
<span>Sept 2018 - Dec 2018</span>
229
+
<divclass="content">
230
+
<h3>Hospitalization Data Mining - Course Project</h3>
231
+
<div>
232
+
<bclass="italic-text">
233
+
</div>
234
+
</div>
235
+
</li>
236
+
<li>
237
+
<span>Apr 2018 - May 2018</span>
238
+
<divclass="content">
239
+
<h3>Web Crawler and Search Engine - Course Project</h3>
<h3>Started Master of Computer Science - Southern Methodist University</h3>
288
+
</div>
289
+
</li>
290
+
<li>
291
+
<span>May 2017</span>
292
+
<divclass="content">
293
+
<h3>Graduated Bachelor of Software Engineering - Qingdao University</h3>
294
+
</div>
295
+
</li>
296
+
<li>
297
+
<span>Feb 2017 - Sept 2017</span>
298
+
<divclass="content">
299
+
<h3>Road Construction Management App - Android Developer @ City Construction Company</h3>
300
+
<div>
301
+
<bclass="italic-text">* Company: The largest public construction company in my hometown Qingdao with 91 million dollars total assets
302
+
</div>
303
+
<div>
304
+
<bclass="italic-text">* Users: 10+ construction teams and supervisors
305
+
</div>
306
+
<div>
307
+
<bclass="italic-text">* Functions: Draw lines or marks and pin icons of machine or labor on a 2D map API(Baidu Map API); update each operation and status of machine or labor then calculate and report daily cost in analytic plots; with different four types of permissions accessing individually
308
+
</div>
309
+
<div>
310
+
<bclass="italic-text">* Duties: Implemented layouts, activities, customized widgets, screen adaption, data synchronization and HTTP request/respond between server and users based on Java
311
+
</div>
312
+
<div>
313
+
<bclass="italic-text">* Gains: SVN Version control, How to customize widget, UML is important, Debugging and testing, Prototype Demo Tool is a saver to know client's demand better, code/material reusability, comment is important
314
+
</div>
315
+
<div>
316
+
<bclass="italic-text">* Self assessment: Fresh try in Android after academic classes, having fun and learning stuffs. Switching between Google and coding all the time, however make it work finally anyway. It doesn't look professionally pretty in graphic, but it works functionally well and indeed saves money for the company
317
+
</div>
318
+
</div>
319
+
</li>
320
+
<li>
321
+
<span>2016</span>
322
+
<divclass="content">
323
+
<h3>Preparing TOFEL and GRE to study in the US</h3>
324
+
</div>
325
+
</li>
326
+
<li>
327
+
<span>Oct 2015</span>
328
+
<divclass="content">
329
+
<h3>1st Cash Prize of Hackthon - Sponsored School Event</h3>
330
+
</div>
331
+
</li>
332
+
<li>
333
+
<span>May 2015 - May 2016</span>
334
+
<divclass="content">
335
+
<h3>Remote Sensing Image Processing for City Planning - Lab Collaboration</h3>
336
+
</div>
337
+
</li>
338
+
<li>
339
+
<span>Apr 2015</span>
340
+
<divclass="content">
341
+
<h3>2nd Coding Challenge - National Coding Challenge</h3>
342
+
</div>
343
+
</li>
344
+
<li>
345
+
<span>May 2014 - May 2015</span>
346
+
<divclass="content">
347
+
<h3>Remote Sensing Image Processing for Coastline Extraction - Lab Collaboration</h3>
348
+
</div>
349
+
</li>
350
+
<li>
351
+
<span>Aug 2013</span>
352
+
<divclass="content">
353
+
<h3>Started Bachelor of Software Engineering - Qingdao University</h3>
0 commit comments