Video Analysis Module for Tellus AI Assessment Platform

Capstone Project

Supervisor: Irwin King, Professor, CUHK | Sep. 2022 - April 2023

Product Introduction | Website

• Designed and constructed E-R diagrams, UML class diagrams, and database schema based on use cases.
• Integrated Microsoft Azure AI Face Service and Python libraries/frameworks (e.g., OpenCV, MediaPipe, DeepFace) to facilitate facial, emotion, accessory, and body language analysis, boosting analysis accuracy by 30%.
• Utilized React, TypeScript, and Material UI to create real-time camera calibration feature and visualization dashboard.
• Developed algorithms using Python for interviewee presentation skills auto-evaluation and cheating detection.
• Designed scoring & commenting algorithms to achieve 93% analysis accuracy and 40% faster runtime.
• Designed 100+ unit and integration tests using Pytest for application testability and security, keeping 80% code coverage.
• Employed REST APIs for video management and implemented CURD operations on evaluation results using MySQL.

Technologies

Python Microsoft Azure Machine Learning Computer Vision Video Analysis Software Engineering System Design React Material UI Node.js MySQL JavaScript/TypeScirpt Git REST API

Deep Learning-Based Time Series Analysis and Forecasting

Summer Research Intern & Term-time Student Helper, CUHK Reliable Computing Lab

Supervisor: Qiang Xu, Professor, CUHK | May 2022 - Oct. 2022

• Built a large-scale repository for time-series analysis and forecasting on GitHub, including different state-of-the-art deep learning models
• Collaborated on creating a systematic pipeline to utilize and evaluate time series prediction models, implementing unified data preprocessing procedures, model instantiation interfaces, and standardized evaluation procedures
• Reproduced and implemented various spatio-temporal traffic prediction models and dataset loaders; modified the models to be compatible with repository interfaces, configurable, and adaptable to different datasets and tasks
• Implemented model evaluations and comparisons to ensure the characteristics and the original results of the models are well preserved

Technologies

Python PyTorch Linux Machine Learning Deep Learning Time-series Forcasting Anaconda Git

Simultaneously Deblurring and Segmentation for Cataract Surgical Instruments

Summer Research Intern, Surgical Robotics and Instrumentation Lab, CUHK

Supervisor: Shing Shin Cheng, Assistant Professor, CUHK | June 2021 - Sept. 2021

• Worked on a spatio-temporal information-based network for cataract surgical instrument segmentation, consisting of a deblurring module and a subsequent segmentation module, with PyTorch and TensorFlow frameworks
• Improved the performance of the segmentation network by incorporating leading-edge deep learning modules
• Designed and conducted experiments on model evaluation and comparison, successfully proved the strength of the network under different special scenarios
• Designed and constructed a cataract surgical instrument dataset for integrating both deblurring and segmentation procedures to demonstrate the holistic novelty of the network
• Investigated the background and current state of research in the field of surgical skill assessment, proposed the motivation and the future research directions for the project

Technologies

Python TensorFlow PyTorch Docker Linux Anaconda LaTeX Machine Learning Deep Learning Computer Vision