Whilst cutting RCs for the templates previously we have found gaps in our automation that need to be resolved. The current validation checks run on ADO are pretty simple (found in npm-build-steps.yml), it just installs yarn dependencies and attempts to build. This should be beefed up to not only validate that a bot can be generated as expected, but that it installs all dependencies in applicable permutations and passes tests end to end.

For Empty or Echo templates found under the /generators directory, we need a pipeline that

• Generates a dotnet webapp bot project
• Builds
• Deploy to Azure
• Test bot

We will be using a clone of the FunctionalTests bits, which should be located in this repo, under the "tests" folder. Associated DevOps pipelines should also be cloned and adjusted to use botframework-components.

Bonus points:

• Dotnet webapp
• Dotnet functions
• JS webapp
• JS functions

Known limitations:

• This is end-to-end from generator to Azure. It does not (cannot) use the exact code that Composer uses to create and build the bot, or provision the Azure resources. This is acceptable for now.
• Issues will spawn from this to cover other Runtime functional/intregration tests, including (but not limited to):
  • Skills matrix
  • BotComponent functionality (see Carlos' tests)
  • Other Runtime configurability, such as CosmosDB, App Insights, stock middleware (feature flags), etc...
  • LUIS
  • QnA
  • Orchestrator
  • Daily SDK builds
  • Testing generators with predefined components