Dawei Wei

Prompt engineering for robot kitchen tasks

1. Built a complete prompt for a robot to execute including

• Language instruction like cutting an apple in 5 pcs
• Lower level vision and action functions like detect, moveto, cut and push
• Code example using these functions like cutting a banana into half

2. Used ChatGPT API to generate correct code using lower level functions

• Feed the prompt into gpt-3.5-turbo, parse the code part and execute it.
• If an error occurs, append the code and the error message to the prompt and feed it again to the api.
• Iteratively doing the above steps until no error occurs.

• Assisted with vehicle testing in simulation and real environment for 3 times
• Arranged control algorithms for autonomous vehicles like S-Curve of Time Optimal Control, Bang-Bang Control

• FEM extension design for 3D Slicer software platform for medical image analysis.
• Feature 1: inputs surface mesh produced from MRI from another extension, creates volume mesh, add material properties (density, Young's modulus, Poisson's ratio) to voxels
• Feature 2: Links OpenSurgSim library to calculate deformation under boundary conditions and external forces, then links VTK library to visualize deformed model
• Application: Imported breast MRI with tumors, simulated deformation from prone to supine pose, performed position error of supine tumor less than 3 cm