You are correct, at the moment the plmxml importer will only load CAD files and not support other formats (fbx, gltf, obj, etc.)
I would write a python file that load your CSV or json
then for each file that is in your csv import the asset into UE: I would use the interchange functions directly, instead of asset import task, Importing Assets Using Interchange in Unreal Engine | Unreal Engine 5.5 Documentation | Epic Developer Community
For other references:
there is code to load datatable or parse folder to load JT files etc.