https://dartcode.org/docs/launch-configuration/ says:

However there are some things you can’t do without a launch configuration, such as:
...
Run Dart (non-Flutter) web apps

But I couldn't see an example of how to do it for a Dart (not flutter) webapp project.

I did get it working by following instructions from this comment to have a launch config like this:

{
  "version": "0.2.0",
  "configurations": [
    {
      "name": "webmidi_bug_demo",
      "request": "launch",
      "type": "dart",
      "program": "web/index.html",
    }
  ]
}

That then when I did "F5" launched the app and launched Chrome just as it does when I run webdev serve --debug manually.

So it seems the trick is to know to set program to set your html file entry point (instead of the dart file as you would for say cli apps). It might be good to update the docs page on dartcode.org about this to help future dart webapp devs use this nice quality of life feature in Dartcode 🙂