After two days struggling with this, I finally made it working.
As @Cody.Albert said, basically you need two pairs of things: MyListViewVM and MyListView, MyListItemVM and MyListItem.
1.1 In MyListViewVM, I have a TArray<MyListItemVM*> MyArray
1.2 Bind MyListViewVM.MyArray to MyListView → Set List Items
2.1 In MyListItem, I setup my view bindings, set the Viewmodel Creation Type to Manual.
2.2 In MyListItem’s blueprint, I implement the IUserObjectListEntry interface.
2.3 Then implement the OnListItemObjectSet event, cast the List Item Object(This is where MyListItemVM* get passed in) to MyListItemVM and set to MyListItemVM variable. (Manually create the ViewModel.)
Let me know if there is a better or “correct” way to do this.