Refactor dashboard navlets to use HTMX by Simrayz · Pull Request #3634 · Uninett/nav

Simrayz · 2025-10-29T09:51:34Z

Scope and purpose

Resolves #3635.

This PR refactors the dashboard navlets logic from a complicated JS flow, to use HTMX where possible, and simplifies necessary javascript plugin interoperability.

See issue #3635 for a full description of changes.

This pull request

Implements loading and updating of navlets with HTMX
Replaces navlet_controller and navlets_controller with navlets_htmx_controller
Renders entire dashboard on initial load of the page, with individual data fetching for each navlet
Simplifies refresh of image- and ajax reload navlets

How to test

Loading navlets

Navlet Load

Open your default dashboard
Verify that navlets have a Spinner that indicates their loading state
Verify that all navlets load and resolve in their "VIEW" mode

Adding and Removing Navlets

Create a new dashboard. Verify that the dashboard has a "No widgets"
Add a new Navlet to your dashboard.
1. Verify that the Navlet is rendered in the leftmost column at the top, and that the "No widgets" message is removed.
2. Verify that the navlet is highlighted, and that the highlight is removed on hover.
3. Add another Navlet to your dashboard.
4. Reload the page, and verify that positions have been saved.
Delete a Navlet.
1. Verify that it is removed from the dashboard.
2. Remove all remaining navlets, and verify that a "No widgets" message is added to the left-most column.

Dashboard Layout

Re-ordering Navlets

Open a dashboard with many navlets, e.g. your Default dashboard.
Hold the drag handle, and move a navlet into a different column.
Reload the page, and verify that the positions were saved.

Updating columns

Open a dashboard with many navlets, e.g. your Default dashboard.
Open dashboard settings, and change the column count.
Verify that the dashboard is reloaded, and the column count is changed.

Compact Dashboard Mode

Open a dashboard with many navlets, e.g. your Default dashboard.
Change the column density from "Normal" to "Compact"
Verify that the spacing between navlets is removed.

Editing Navlets

Fill in section about how to test editing of navlets.

Periodic Refresh of Navlets

Refresh with standard interval

This can be tested with all Navlets that have a refresh_interval defined, and don't have ajax_reload or image_reload set to True. I have selected RoomStatus as an example as the refresh interval is short.

Add a RoomStatus ("Rooms with active alerts") navlet to a dashboard.
Take note of the "Last update" time, and wait 30 seconds.
Verify that the Navlet is reloaded. You can also view this in the Network tab in dev tools.

Refresh with `ajax_reload=True`

Create an empty dashboard.
Add an Alert, PDU and UPS navlet.
Configure the navlets to show content.
The default refresh interval is 30 seconds, so open the network tab and look for repeating render requests.
Verify that the "Last update" text in the bottom-right is updated every thirty seconds. Does not work for the Alert widget, if there is no data available in graphite.

Refresh for VlanGraphNavlet and GraphWidget

Add a Vlan Graph to a dashboard.
1. Go into edit mode, and select a Vlan. Not all of them have an URL, so try one that works, e.g. VLAN 20 for employees.
2. Verify that an image (or placeholder with No Data) is rendered.
Add a Chart (GraphWidget) to a dashboard.
1. Go into edit mode. You need a Chart URL and a Target URL. I found the most practical way to find values for this form was to find a GraphWidget in the account_navlet table, and copy the values from preferences.
2. Verify that an image (or placeholder with No Data) is rendered.

Contributor Checklist

Every pull request should have this checklist filled out, no matter how small it is.
More information about contributing to NAV can be found in the
Hacker's guide to NAV.

github-actions · 2025-10-29T10:00:49Z

Test results

27 files 27 suites 45m 17s ⏱️
2 805 tests 2 804 ✅ 0 💤 0 ❌ 1 🔥
20 824 runs 20 822 ✅ 0 💤 0 ❌ 2 🔥

For more details on these errors, see this check.

Results for commit b92a2a9.

♻️ This comment has been updated with latest results.

codecov · 2025-10-29T10:01:05Z

Codecov Report

❌ Patch coverage is 84.44444% with 14 lines in your changes missing coverage. Please review.
✅ Project coverage is 63.14%. Comparing base (317ca31) to head (7863e4d).
⚠️ Report is 22 commits behind head on master.

Files with missing lines	Patch %	Lines
python/nav/web/navlets/vlangraph.py	0.00%	4 Missing ⚠️
python/nav/web/navlets/__init__.py	93.61%	3 Missing ⚠️
python/nav/web/navlets/graph.py	0.00%	3 Missing ⚠️
python/nav/web/navlets/status2.py	50.00%	2 Missing ⚠️
python/nav/web/navlets/feedreader.py	0.00%	1 Missing ⚠️
python/nav/web/navlets/report.py	50.00%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #3634      +/-   ##
==========================================
+ Coverage   63.04%   63.14%   +0.10%     
==========================================
  Files         614      614              
  Lines       45457    45502      +45     
  Branches       43       43              
==========================================
+ Hits        28660    28734      +74     
+ Misses      16787    16758      -29     
  Partials       10       10

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

hmpf

Add one commit that fixes the get's and post's of all the TemplateView desecendants to also have *args (or *_) and **kwargs (or **__) even if they don use them. Devs used to Django expect it.

python/nav/web/navlets/__init__.py

hmpf · 2025-10-31T07:31:25Z

python/nav/web/navlets/__init__.py

    @property
    def mode(self):
        """Fetch the mode of this request"""
        return self.request.GET.get('mode', NAVLET_MODE_VIEW)


Changing viewclass.as_view() to spit out two different classes depending on mode might be better, but is out of scope. Django-style would be to have the mode as part of the url (`path("something/str:mode", ..) and two different views but that's even more out of scope.

I agree. The Navlets code is quite strange, and really we should separate GET and POST at least. I opted not to dig too much into the Navlets in this round, and leave that for later.

python/nav/web/navlets/graph.py

python/nav/web/navlets/vlangraph.py

hmpf · 2025-10-31T07:40:43Z

python/nav/web/templates/navlets/_add_navlet_modal.html

+                  hx-post="{% url 'add-user-navlet' dashboard_id %}"
+                  hx-swap="beforeend"
+          >


What happens if you keep action and post in addition to what's here now?

The native form submit event is overruled by HTMX, so adding them would make no difference. Action is only used for html form submission, or by certain javascript submit handlers across the codebase.

Q: Is progressive fallback not a goal here? (not that I'm sure this ever had functional progressive fallback).

johannaengland

When I set the column number to one I get a different behavior
Before:

After:

(I double checked that for both the column number is set to one)

When I try to add a "Rooms with active alerts" or "Locations with active alerts" widget this is all I get:

python/nav/web/navlets/__init__.py

python/nav/web/webfront/views.py

tests/integration/web/webfront_test.py

tests/integration/web/navlets_test.py

Simrayz · 2025-12-03T10:39:50Z

@johannaengland Thanks for finding that bug! The problem was a missing css class when the column count is 1 🙃 It should now behave as normal:

johannaengland

The tests are failing due to the change in the column change message

I also still cannot get the "Rooms/Locations with active alerts" widget to work

And I found a fun new bug, when logging in as a non-admin user and creating a new dashboard, I get the following view:

Simrayz · 2025-12-05T10:05:44Z

@johannaengland I fixed the authentication error by adding the dashboard loading URL to the authentication middleware whitelist. I also fixed the failing test.

The Room/Location Active Alert navlets seem to work for me.

johannaengland · 2025-12-08T13:26:28Z

python/nav/web/auth/utils.py

        '/index/audit-logging-modal/',
        '/refresh_session',
    ]
+    auth_not_required_regex = [r'^/index/dashboard/[^/]+/load/?$']


Are you sure this is the best way to fix this? Because you're mentioning to fix authentication for unauthenticated users, but my problem was authentication for non-admin users

The fix was intended to allow anonymous users to load the default dashboard, as only the ("") view is whitelisted. I have tested with an admin and a non-admin user (but authenticated), and both can load a dashboard they own or that is shared.

However, the views to render navlets are not added to the authorization whitelist, so the problem might lie elsewhere... Are you aware of why 'get-user-navlet' is allowed for unauthenticated users?

I think it is because the dashboard that belongs to default user should also be visible for when a user is not logged in yet

lunkwill42 · 2025-12-10T15:23:55Z

Go into edit mode. You need a Chart URL and a Target URL. I found the most practical way to find values for this form was to find a GraphWidget in the account_navlet table, and copy the values from preferences.

A more practical method might be to just find any interactive graph, say the activity graphs for any port on any switch and click the graph's "Add graph to dashboard" button :)

lunkwill42

I'm not deeply familiar with how the dashboard/navlet system works to begin with, but this certainly does look like a massive improvement over the existing code. Therefore, I don't have terribly much to say about the code itself (apart for my usual test naming nitpicks 😆), but I have tested in my browser (Firefox 145) and found a couple of weird things:

Dashboard compact mode looks different than before, and has issues. Something funny happens with the internal padding of widgets, and there are gaps in the vertical space between widgets where there before were none.
I added widgets of type "Rooms with active alerts" or "Locations with active alerts", then expanded (accordion?) one of the rooms or locations with active alerts to see the list of alerts under each. As soon as the widget is refreshed (on the refresh interval), my expansion is immediately forgotten and I am returned to the default state of the widget. This does not appear to happen in the original version, and this definitely reduces usability. I have no idea how this was achieved in the original.
There are non-fixup commits that exist only to refactor code that was added previously in the same PR. While refactoring is fine, doing it within a single PR just makes it harder to review, so I would suggest squashing things into more logical commits when you get to the "squashing fixups"-stage.
There are some comments about issues with rendering for unauthenticated users. I haven't the time or the brainpower to grok the issue this late in the afternoon, I'll have to get back to it 😆

lunkwill42 · 2025-12-10T14:05:53Z

python/nav/web/navlets/__init__.py

+        :param override_mode: Optional\; if provided, overrides the mode (VIEW or EDIT)
+            sent in the request. If None, uses self.mode. Used to enable the navlet to
+            return the correct template in error situations.
+        :type override_mode: str or None


non-blocking: While you see this all over old NAV code, please recall that this was written before type annotations were a thing in Python (some IDEs would even parse docstrings like these to provide type hints).

I would much prefer that type annotations were used for this rather than docstrings for new/rewritten/refactored code, in this modern day and age 😄

I don't know that type annotations can be used to give parameter explanations, though, so this can be kept in the docstring.

lunkwill42 · 2025-12-10T14:16:05Z

python/nav/web/templates/navlets/_add_navlet_modal.html

+                  hx-post="{% url 'add-user-navlet' dashboard_id %}"
+                  hx-swap="beforeend"
+          >


Q: Is progressive fallback not a goal here? (not that I'm sure this ever had functional progressive fallback).

tests/integration/web/navlets_test.py

tests/integration/web/webfront_test.py

tests/integration/widget_test.py

tests/integration/web/webfront_test.py

Simrayz · 2026-02-03T14:34:33Z

Fun new bug I just found, please have a test and see if you can reproduce it:

When I add a completely new dashboard and add at least two widgets I am not able to move the widgets. But if reload the page then it works

Weird bug yeah, it seems like jQuery UI did not initialize sorting properly unless there are existing navlets. It is fixed now to init on every new navlet.

johannaengland

https://media2.giphy.com/media/v1.Y2lkPTc5MGI3NjExYTUxM3FobHA4ZHlibXJjNGEyc3B6eHZ4dzFucjJuZDFvNDA4NDN5YyZlcD12MV9pbnRlcm5hbF9naWZfYnlfaWQmY3Q9Zw/whhCAcuxtAXcrQhHTk/giphy.gif

lunkwill42 · 2026-02-05T09:50:52Z

I'm sure @johannaengland and @hmpf are adequately able to test this feature, but I never did get an answer to my question above:

Q: Is progressive fallback not a goal here? (not that I'm sure this ever had functional progressive fallback).

Also, I see SonarQube is not happy, it would be nice if someone took the time to review and then fix or dismiss its warnings.

sonarqubecloud · 2026-02-05T11:18:52Z

Quality Gate passed

Issues
1 New issue
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
1.1% Duplication on New Code

See analysis details on SonarQube Cloud

Simrayz self-assigned this Oct 29, 2025

Simrayz added cleanup javascript htmx labels Oct 29, 2025

Simrayz added the modernisation label Oct 29, 2025

Simrayz force-pushed the chore/refactor-dashboard-navlets-to-use-htmx branch 4 times, most recently from 577b2c8 to 79f0b28 Compare October 30, 2025 12:34

Simrayz requested a review from a team October 30, 2025 14:21

Simrayz force-pushed the chore/refactor-dashboard-navlets-to-use-htmx branch 2 times, most recently from c9d6286 to 20865d9 Compare October 30, 2025 14:43

hmpf requested changes Oct 31, 2025

View reviewed changes

Simrayz force-pushed the chore/refactor-dashboard-navlets-to-use-htmx branch 2 times, most recently from 4418f4c to 143c621 Compare November 3, 2025 09:32

Simrayz marked this pull request as ready for review December 1, 2025 09:23

johannaengland requested changes Dec 2, 2025

View reviewed changes

Simrayz requested review from hmpf and johannaengland December 3, 2025 10:40

johannaengland requested changes Dec 4, 2025

View reviewed changes

Simrayz requested a review from johannaengland December 5, 2025 10:09

johannaengland reviewed Dec 8, 2025

View reviewed changes

Simrayz force-pushed the chore/refactor-dashboard-navlets-to-use-htmx branch from 2eb117e to dd77674 Compare December 9, 2025 17:56

lunkwill42 requested changes Dec 10, 2025

View reviewed changes

Simrayz force-pushed the chore/refactor-dashboard-navlets-to-use-htmx branch from dd77674 to 6b4f70e Compare December 11, 2025 11:01

johannaengland approved these changes Feb 4, 2026

View reviewed changes

Simrayz added 19 commits February 5, 2026 10:03

Add initial support for rendering dashboard with htmx

e7580a1

Add support for adding and removing navlets with htmx

86af992

Add loading spinner to navlets

272da10

Change column chooser to update with htmx

79483af

Move navlet listeners to htmx controller

71de88d

Add support for saving navlet forms with HTMX

da83c55

Handle form errors in status navlet

de0590d

Support re-rendering of form select elements with htmx

3200829

Add bust parameter to graph urls

e3a3697

Add refresh template for handling periodic refresh

4d68c15

Refactor navlets htmx controller

a5f2b73

Refactor navlet handlers in separate plugin

5b7c3c5

Add changelog entry

1191fc6

Support ajax reload with htmx swap

98065fe

Remove old navlet controllers

464f6a6

Add dashboard loading tests

e014c18

Add navlet tests

278859f

Fix authentication error for unauthenticated users

8418554

Refactor navlets htmx controller and fix initialization bug

e8c8ce4

Simrayz force-pushed the chore/refactor-dashboard-navlets-to-use-htmx branch from 5d9654e to b92a2a9 Compare February 5, 2026 09:26

hmpf approved these changes Feb 5, 2026

View reviewed changes

Simrayz added 2 commits February 5, 2026 12:17

Fix bug where active accordion tab is not preserved when refreshing

5aaac14

Fix bug where navlets take up more vertical space than needed

7863e4d

Simrayz force-pushed the chore/refactor-dashboard-navlets-to-use-htmx branch from b92a2a9 to 7863e4d Compare February 5, 2026 11:17

Simrayz merged commit 40e206f into master Feb 5, 2026
12 of 14 checks passed

Simrayz deleted the chore/refactor-dashboard-navlets-to-use-htmx branch February 5, 2026 11:18

Conversation

Simrayz commented Oct 29, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Scope and purpose

This pull request

How to test

Loading navlets

Navlet Load

Adding and Removing Navlets

Dashboard Layout

Re-ordering Navlets

Updating columns

Compact Dashboard Mode

Editing Navlets

Periodic Refresh of Navlets

Refresh with standard interval

Refresh with ajax_reload=True

Refresh for VlanGraphNavlet and GraphWidget

Contributor Checklist

Uh oh!

github-actions bot commented Oct 29, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Test results

Uh oh!

codecov bot commented Oct 29, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

hmpf left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

hmpf Oct 31, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

johannaengland left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Simrayz commented Dec 3, 2025

Uh oh!

johannaengland left a comment

Choose a reason for hiding this comment

Uh oh!

Simrayz commented Dec 5, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

lunkwill42 commented Dec 10, 2025

Uh oh!

lunkwill42 left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Simrayz commented Oct 29, 2025 •

edited

Loading

Refresh with `ajax_reload=True`

github-actions bot commented Oct 29, 2025 •

edited

Loading

codecov bot commented Oct 29, 2025 •

edited

Loading

hmpf Oct 31, 2025 •

edited

Loading