It turns out that there is no uninstall mechanism provided by Mac OS for software installed via .pkg installers. If it happened to be an application, then you can drag its icon into Trash, and most of its files will be deleted. But if it's something else which doesn't have an icon then there is no easy way to uninstall it.
I found two methods in the end.
One is an application called UninstallPkg (it costs $10). It seems to work fine, and is a lot more convenient then the manual method outlined below.
The manual command line method is as follows (based on this post).
First, get the list of installed packages with
Find the name of the package you want to uninstall, and list its files with
This list should help you locate the package files in the filesystem, and then you can delete them using your favourite method. Packages appear to install under
/Library for the most part.