Couldn't find any writeup for this specifically elsewhere, so here's how:

  1. Go to Settings -> Apps -> Choose default apps -> Opening links
  2. In the Installed apps list, tap the app you no longer want to be the default for something
  3. In the ensuing dialog, tap Clear defaults

The above worked on a Samsung Galaxy Tab S7 Wi-Fi.