About Me

Hi there! I'm Jeff Liu. My passion is in AI applications, especially household robotics. I have a dream that one day, robots and automated machines will handle all repetitive tasks, allowing people to enjoy their lives and focus on what they truly love.

My interests include:
•   Theory Topics: Reinforcement Learning, Robotics, AI Agent
•   Applications: Household Robot, Smart Glove, AI Agent in taxation field
•   Social Justice: AI Ethics, Income Equality in AI Era

Projects

Color Calibration

A light condition adaptive color calibration project with full MLOps process. The system is designed to recognize the color in any light conditions, with a high robust. Use different models to evaluate validation.  [->GitHub]
MLOps ML ETL ETV MTL MRD RandomForest GBT
small neural network
 

Value-based Reinforcement Learning Methods on Breakout Game

Explore designing and implementing value-based methods on breakout game. [->GitHub]
Q-learning DQN RL CUDA
 

Multi-Functional Cleaning Robot

This project has launched both in software and hardware fronts.  I build a turtlebot3 robot, with functions I am developing a smart cleaning robot with frontier-based exploration, path planning, route following and voice control. [->GitHub]
Map Processing Path Planning
GUI Control Voice Control
 

IoT Platform (Collaborated Project)

This project builds a scalable IoT platform for real-time monitoring and control of diverse devices. Using MQTT and a Raspberry Pi hub, it supports remote setup, data display, and rule-based automation. With contributions from Aby Iberkleid, James Petullo, and Shubham Kaushik.
MQTT ROS React Edge AI
 

Policy-based Reinforcement Learning Methods on Walker2d task

Explore designing and implementing policy-based methods on walker2d task. [->GitHub]
Policy-Gradient Control
Actor-Critic PPO RL
 

Following Line with Color-based Centroid Detection

This program can allow robot following a line and avoid obstacles.  The following logic is based on color and centroid detection. [->GitHub]
ROS1 OpenCV PID Control
 

Agent Kit

A general-purpose AI Agent framework, desgined for seamless extension into domain-specific fields like Finance and Healthcare.
 [->GitHub]
AI Agent AI Infra RAG
Tool-augmented Reasoning
Prompt Engineering
 
Escape the Maze with Left Wall Following

Escape the Maze with Left Wall Following

This is my first project and designed at Brandeis Robotics lab.  This project is to escape the maze with left wall following algorithm. [->GitHub]
ROS1 Linux LiDAR
 

Smart Glove

An AI-powered and raspberrypi-pico based smart glove that can control the robot arm. [->GitHub]
Embedded Development
Gesture Recognition
 

Solve Blackjack - Monte Carlo ES

The game blackjack was simulated 2 million times using the Monte Carlo ES method and converged to the optimal solution. [->GitHub]
Reinforcement Learning
Monte Carlo ES
 
Fiducials Detection Demo

Fiducials Detection

Implemented a real-time ArUco fiducial marker detection system using computer vision techniques. [->GitHub]
Computer Vision ArUco Markers Fiducials
 
Simple AI Agent Demo

Intelligent NPC Demo

This is the prototype or demo of project "Work With Farmer Louise". The NPC know nothing at first, all skills need to be learned from player. [->GitHub]
OpenAI API Prompt Engineering Pygame
 
A Star Demo

A* Route Planner

Use open source map to implement A* route plan.  The start position is Tufts University,  and the end point is in a park near Mystic River.  No Demo, See Source Codes:  [->GitHub]
OSM C++ A*
 

Ichiban Lottery (Commercial Project)

The Tianyuan Ichiban Lottery is a WeChat Mini Program offering anime figure lottery products via a digital platform system. Deployed in real market and attracted real users, the program generated over $15,000 in revenue.

Research Notes

Notes and Tutorial Resources

  •  

    Economics Studies Notebook

    The notes of Economics, especially Quantitative Economics.
    Focus on Macroeconomics, Quantitative Economics, and AI applications in economics.

  •  

    Reinforcement Learning (Chinese Notes and Tutorials, ONGOING)

    The study notes of Reinforcement Learning
    A video explanation series will be released later on Bilibili.
    Mainly based on Sutton&Barto's textbook.
    This series is suitable for those who has limited background on Computer Science and Mathmatics, but interested in Reinforcement Learning and don't know how to learn it.

  • Deep Learning (Chinese Notes and Tutorials, ONGOING)

    The study notes of Deep Learning
    A video explanation series will be released later on Bilibili.
    Mainly based on Ian's textbook.
    This series is suitable for those who has limited background on Computer Science and Mathmatics, but interested in Deep Learning and don't know how to learn it.

  • AP CSA Open Course

    To be announced...

  • Leetcode Coding Series

    To be announced...

  • The Golden Rules

    The golden rules of one's life and living in current society.
    The Golden Rule encapsulates my entire worldview, value system, and philosophy of life. It reflects the ethical foundation upon which I live and act.

  • Reading Notes (Economics Theory, Social Science, Psychology Studies)

    Reading notes of selected books.
    A video explanation series will be released later on Bilibili.
    Explore more domains, instead of techniques only.