Skip to content

Commit 6f93c7b

Browse files
lunkwill42lunkwill42
authored
Merge pull request #3420 from Uninett/doc/hacking-devcontainers
Document practical usage of devcontainer
2 parents 388145c + 9396e99 commit 6f93c7b

6 files changed

Lines changed: 228 additions & 0 deletions

File tree

‎Makefile‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -23,6 +23,13 @@ docclean:
2323
-rm -rf doc/_build
2424

2525
doc: doc/reference/alerttypes.rst
26+
sphinx-build doc/ build/sphinx/html/
27+
28+
docwatch:
29+
@while inotifywait -e modify -e move -e create -e delete -r --exclude \# doc/ NOTES.rst; \
30+
do \
31+
sphinx-build doc/ build/sphinx/html/ ;\
32+
done \
2633

2734
doc/reference/alerttypes.rst: .FORCE
2835
python3 doc/exts/alerttypes.py > $@

‎changelog.d/3398.added.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
Document practical usage of devcontainer for developers

‎doc/hacking/hacking.rst‎

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -504,6 +504,21 @@ of these are automatically run every time a change is pushed to a branch
504504
in the offical NAV Github repository, or when a pull request is opened
505505
against this repository.
506506

507+
Development environments
508+
========================
509+
510+
A complete NAV development environment has many dependencies, and not all are
511+
necessarily easy to satisfy on an arbitrary developer's computer. We have
512+
therefore built two different development environment specifications based on
513+
Docker Compose, which should make running NAV for development smoother.
514+
515+
1. The oldest of these is a plain Docker Compose specification. Read
516+
:doc:`using-docker` for further instructions.
517+
2. The newest of them is a `devcontainers`_ specification, which should
518+
integrate more smoothly into most popular IDEs for Python developers (such
519+
as PyCharm or VS Code). Read :doc:`using-devcontainers` for further
520+
instructions.
521+
507522
Tips and tricks
508523
===============
509524

@@ -551,3 +566,4 @@ __ Github_
551566
.. _pre-commit: https://pre-commit.com/
552567
.. _Ruff: https://docs.astral.sh/ruff/
553568
.. _SonarCloud: https://sonarcloud.io/
569+
.. _devcontainers: https://containers.dev/

‎doc/hacking/index.rst‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,7 @@
77

88
hacking
99
hacking-with-pycharm
10+
using-devcontainers
1011
using-docker
1112
javascript
1213
release-procedure

‎doc/hacking/using-devcontainers.rst‎

Lines changed: 202 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,202 @@
1+
=======================================
2+
Using devcontainers for NAV development
3+
=======================================
4+
5+
NAV provides a `devcontainer <https://containers.dev/>`_ definition to simplify
6+
the process of creating a complete environment for running and developing
7+
NAV. This should integrate smoothly with popular IDEs like PyCharm and VS
8+
Code. There are also command line tools to interact with devcontainers for
9+
those who prefer not to use IDEs for development, but we don't yet have any
10+
experience with these.
11+
12+
What is provided by the NAV devcontainer
13+
========================================
14+
15+
The devcontainer definition resides in the :file:`.devcontainer/` directory and
16+
uses `Docker Compose`_ to provide three containers:
17+
18+
1. The main devcontainer, ``nav``, which should be provisioned with all
19+
dependencies normally required to build and run NAV from source code.
20+
2. A PostgreSQL service container (``db``) to provide relational database
21+
services to NAV.
22+
3. A Graphite service container (``graphite``) to provide both the back-end
23+
(``carbon``) and front-end (``graphite-web``) components that provide time
24+
series database services to NAV.
25+
26+
Working efficiently with the ``nav`` devcontainer
27+
=================================================
28+
29+
Tools like PyCharm and VS Code should automatically detect the devcontainer
30+
definitions when you open the NAV project in them. They should typically prompt
31+
you and ask if you want to start the devcontainer and re-open the IDE inside
32+
it.
33+
34+
The devcontainer only provides an environment for development, it doesn't do
35+
much magic in the background.
36+
37+
TL;DR
38+
-----
39+
40+
1. Open a terminal inside the devcontainer. Issue these commands to start the
41+
web server::
42+
43+
navsyncdb
44+
django-admin runserver
45+
46+
2. Open a second terminal inside the devcontainer. Issue this command to start
47+
all NAV background processes::
48+
49+
sudo nav start
50+
51+
3. Open a third terminal inside the devcontainer. Issue this command to
52+
automatically build all CSS stylesheets when the SASS sources change::
53+
54+
make sasswatch
55+
56+
4. Open a fourth terminal inside the devcontainer. Issue this command to
57+
automatically build the HTML documentation output when the source file
58+
change::
59+
60+
make docwatch
61+
62+
63+
Python environment
64+
------------------
65+
66+
The Python environment inside the devcontainer is a *virtualenv* located in
67+
:file:`/home/vscode/.venv`, and the :file:`/home/vscode/.venv/bin/` directory
68+
is automatically pre-pended to the container's :envvar:`PATH` environment
69+
variable, so that any program inside this environment is available and
70+
automatically executed within said environment.
71+
72+
The environment is built at container startup by `uv`_, using the :samp:`uv
73+
sync --all-extras` command. This ensures that all runtime requirements and
74+
development dependency groups mentioned in NAV's :file:`pyproject.toml` should
75+
be installed into the environment.
76+
77+
Using :samp:`uv sync` also ensures that all NAV command line programs are
78+
available on the :envvar:`PATH` search path, and that changes you make to their
79+
source code is immediately reflected when running them. If you add new NAV
80+
command programs, however, you may need to run :samp:`uv sync --all-extras`
81+
over again, to ensure their stubs are installed in the environment's
82+
:file:`bin/` directory.
83+
84+
85+
Configuring NAV
86+
---------------
87+
88+
At startup, the container installs all of NAV's example configuration files
89+
into the :file:`/home/vscode/.venv/etc/nav/` directory. This can be confirmed
90+
at any time by issuing the :samp:`nav config where` command in a terminal
91+
inside the container.
92+
93+
NAV's :file:`db.conf` configuration file is automatically imbued with the
94+
options necessary to let NAV connect to the PostgreSQL server in the ``db``
95+
container.
96+
97+
The container also provides common text editors like :program:`vim` and
98+
:program:`nano`, which should enable you to edit the configuration files if
99+
necessary.
100+
101+
102+
Preparing the database schema
103+
-----------------------------
104+
105+
When you start the devcontainer for the first time, the database may be
106+
completely empty. You will need to run the :samp:`navsyncdb` command in order
107+
to initialize and/or migrate NAV's database schema, before any NAV programs are
108+
usable within the container.
109+
110+
111+
Running the NAV web interface
112+
-----------------------------
113+
114+
When developing, the NAV web interface is best served by the built-in Django
115+
development web server, using this command in a terminal: :samp:`django-admin
116+
runserver`.
117+
118+
This server will serve on port *8000* inside the container. After running this
119+
command, your IDE may prompt you to forward this port to your host machine, so
120+
you will be able to browse the web site from your desktop browser (in some
121+
cases, it may automatically forward port 8000 also to your localhost).
122+
123+
Logging in to the NAV web interface
124+
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
125+
126+
The ``admin`` user ships with the default password ``admin``. If you need to
127+
test with unprivileged users, more can be added using the *User and API
128+
Administration* tool in the toolbox.
129+
130+
User accounts and passwords can also be manipulated on the command line using
131+
the :program:`navuser` program (see :ref:`navuser usage docs <navuser-usage>`
132+
for details).
133+
134+
135+
Running NAV programs
136+
--------------------
137+
138+
NAV command line programs (those specified by the ``project.scripts`` section
139+
of :file:`pyproject.toml`) are all available on :envvar:`PATH` and can be run
140+
directly if need be. The :program:`nav` process control command is also
141+
available, so that all background daemons and cron jobs can be started by
142+
issuing the :samp:`sudo nav start` command.
143+
144+
During certain development tasks, it may be preferable to manually run specific
145+
daemon programs in the foreground rather than using the :program:`nav` command,
146+
e.g. :samp:`ipdevpolld -f -s` instead of :samp:`nav start ipdevpolld`.
147+
148+
149+
(Re)building CSS stylesheets from SASS sources
150+
----------------------------------------------
151+
152+
If you make changes to the SASS definitions, you will have to execute
153+
:samp:`make sassbuild` to rebuild the CSS assets that are served by the web
154+
server.
155+
156+
More effectively, you may want to use the :samp:`make sasswatch` command, which
157+
will monitor all the SASS source files for changes and automatically rebuild
158+
the stylesheets on every change.
159+
160+
161+
(Re)building NAV's Sphinx documentation
162+
---------------------------------------
163+
164+
NAV's documentation sources reside in the :file:`doc/` directory. These can be
165+
manually built into HTML using the :samp:`make doc` command. The output
166+
directory is automatically served by the Django development web server on the
167+
``/doc/`` URI.
168+
169+
More effectively, you may want to use the :samp:`make docwatch` command, which
170+
will monitor all the documentation source files for changes and automatically
171+
rebuild the HTML output on every change.
172+
173+
174+
Installing Python packages manually
175+
-----------------------------------
176+
177+
If you want to install extra Python packages that are not specified in
178+
:file:`pyproject.toml`, use :samp:`uv pip install {package_name}` to do so.
179+
180+
Please be aware, though, that these packages will potentially be removed any
181+
time :samp:`uv sync` is rerun. If a package is a new runtime dependency for
182+
code you're working on, it should be added to the ``project.dependencies`` list
183+
of :file:`pyproject.toml` ASAP. If it's a development tool that is nice or
184+
necessary to have, it should be added to the relevant dependency groups in
185+
the ``dependency-groups`` section of :file:`pyproject.toml`.
186+
187+
188+
PyCharm oddities
189+
================
190+
191+
PyCharm seems to have problems with properly detecting the correct Python
192+
interpreter when running inside the devcontainer. When started, it lists the
193+
project as having *no interpreter*, and the only way to fix it is to manually
194+
select an existing interpreter (specifically,
195+
:file:`/home/vscode/.venv/bin/python`). Unfortunately, this choice does not
196+
seem to be persisted anywhere, so every time PyCharm is re-opened inside the
197+
container, this interpreter selection procedure needs to be repeated.
198+
199+
200+
201+
.. _Docker Compose: https://docs.docker.com/compose/
202+
.. _uv: https://docs.astral.sh/uv/

‎doc/howto/commandline-utilities.rst‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -249,6 +249,7 @@ Usage
249249
~~~~~
250250
.. program-output:: navsynctypes --help
251251

252+
.. _navuser-usage:
252253
------------------
253254
:command:`navuser`
254255
------------------

0 commit comments

Comments
 (0)