Template layout/alert.html:

<div{% if alert.css_id %} id="{{ alert.css_id }}"{% endif %}{% if alert.css_class %} class="{{ alert.css_class }}"{% endif %}>
    {% if dismiss %}<button type="button" class="close" data-dismiss="alert">&times;</button>{% endif %}
    {{ content|safe }}
</div>

Example from Bootstrap 5 docs:

<div class="alert alert-warning alert-dismissible fade show" role="alert">
  <strong>Holy guacamole!</strong> You should check in on some of those fields below.
  <button type="button" class="btn-close" data-bs-dismiss="alert" aria-label="Close"></button>
</div>

Observations:

• If dismiss is True, the div should probably also have alert-dismissible (as well as fade and show) classes? Or is this intended and these classes should be added to the css_class keyword of Alert’s constructor?
• The Close button still uses Bootstrap 4 markup. With Bootstrap 5, the close button’s class should be btn-close (instead of close), data-dismiss="…" should be data-bs-dismiss="…", and button should not have any content (x-mark is a SVG image added via CSS)
• While the button tag is placed below the alert’s content in the Bootstrap example, placing the button tag above the content also seems to work.