Question: :dev branch container gives permission denied errors on migration script (in /app) even after new install #1002
Description
Hi there,
I've been using ArchiveBox for quite a while, previously I was using the :latest branch, which works fine for most things, except that SingleFile doesn't work because of the missing link to /usr/bin/chromium-browser
I see there's a fix in the dockerfile, so when I switch to the dev docker release the link is correct however I cannot start the server due to permission errors, this happens regardless of PUID, PGID host permissions (777 for debugging) of the data folder, the issue appears to be for some reason the /app/ folder has the wrong permissions
Initial logs
Attaching to archivebox-archivebox-1
archivebox-archivebox-1 | Change in ownership detected, please be patient while we chown existing files
archivebox-archivebox-1 | This could take some time...
archivebox-archivebox-1 | chown: changing ownership of '/data/index.sqlite3': Permission denied
archivebox-archivebox-1 | chown: changing ownership of '/data/archive': Permission denied
archivebox-archivebox-1 | chown: changing ownership of '/data/logs/errors.log': Permission denied
archivebox-archivebox-1 | chown: changing ownership of '/data/logs': Permission denied
archivebox-archivebox-1 | chown: changing ownership of '/data/sources': Permission denied
archivebox-archivebox-1 | chown: changing ownership of '/data/ArchiveBox.conf': Permission denied
archivebox-archivebox-1 | chown: changing ownership of '/data': Permission denied
archivebox-archivebox-1 | chown: changing ownership of '/data': Permission denied
archivebox-archivebox-1 | [i] [2022-07-24 12:00:30] ArchiveBox v0.6.3: archivebox server --quick-init 0.0.0.0:8000
archivebox-archivebox-1 | > /data
archivebox-archivebox-1 |
archivebox-archivebox-1 | [^] Verifying and updating existing ArchiveBox collection to v0.6.3...
archivebox-archivebox-1 | ----------------------------------------------------------------------
archivebox-archivebox-1 |
archivebox-archivebox-1 | [*] Verifying archive folder structure...
archivebox-archivebox-1 | + ./archive, ./sources, ./logs...
archivebox-archivebox-1 | + ./ArchiveBox.conf...
archivebox-archivebox-1 |
archivebox-archivebox-1 | [*] Verifying main SQL index and running any migrations needed...
archivebox-archivebox-1 | Traceback (most recent call last):
archivebox-archivebox-1 | File "/usr/local/bin/archivebox", line 33, in <module>
archivebox-archivebox-1 | sys.exit(load_entry_point('archivebox', 'console_scripts', 'archivebox')())
archivebox-archivebox-1 | File "/app/archivebox/cli/__init__.py", line 140, in main
archivebox-archivebox-1 | run_subcommand(
archivebox-archivebox-1 | File "/app/archivebox/cli/__init__.py", line 80, in run_subcommand
archivebox-archivebox-1 | module.main(args=subcommand_args, stdin=stdin, pwd=pwd) # type: ignore
archivebox-archivebox-1 | File "/app/archivebox/cli/archivebox_server.py", line 64, in main
archivebox-archivebox-1 | server(
archivebox-archivebox-1 | File "/app/archivebox/util.py", line 114, in typechecked_function
archivebox-archivebox-1 | return func(*args, **kwargs)
archivebox-archivebox-1 | File "/app/archivebox/main.py", line 1280, in server
archivebox-archivebox-1 | run_subcommand('init', subcommand_args=['--quick'], stdin=None, pwd=out_dir)
archivebox-archivebox-1 | File "/app/archivebox/cli/__init__.py", line 80, in run_subcommand
archivebox-archivebox-1 | module.main(args=subcommand_args, stdin=stdin, pwd=pwd) # type: ignore
archivebox-archivebox-1 | File "/app/archivebox/cli/archivebox_init.py", line 43, in main
archivebox-archivebox-1 | init(
archivebox-archivebox-1 | File "/app/archivebox/util.py", line 114, in typechecked_function
archivebox-archivebox-1 | return func(*args, **kwargs)
archivebox-archivebox-1 | File "/app/archivebox/main.py", line 344, in init
archivebox-archivebox-1 | for migration_line in apply_migrations(out_dir):
archivebox-archivebox-1 | File "/app/archivebox/util.py", line 114, in typechecked_function
archivebox-archivebox-1 | return func(*args, **kwargs)
archivebox-archivebox-1 | File "/app/archivebox/index/sql.py", line 143, in apply_migrations
archivebox-archivebox-1 | call_command("makemigrations", interactive=False, stdout=null)
archivebox-archivebox-1 | File "/usr/local/lib/python3.10/site-packages/django/core/management/__init__.py", line 168, in call_command
archivebox-archivebox-1 | return command.execute(*args, **defaults)
archivebox-archivebox-1 | File "/usr/local/lib/python3.10/site-packages/django/core/management/base.py", line 371, in execute
archivebox-archivebox-1 | output = self.handle(*args, **options)
archivebox-archivebox-1 | File "/usr/local/lib/python3.10/site-packages/django/core/management/base.py", line 85, in wrapped
archivebox-archivebox-1 | res = handle_func(*args, **kwargs)
archivebox-archivebox-1 | File "/usr/local/lib/python3.10/site-packages/django/core/management/commands/makemigrations.py", line 182, in handle
archivebox-archivebox-1 | self.write_migration_files(changes)
archivebox-archivebox-1 | File "/usr/local/lib/python3.10/site-packages/django/core/management/commands/makemigrations.py", line 220, in write_migration_files
archivebox-archivebox-1 | with open(writer.path, "w", encoding='utf-8') as fh:
archivebox-archivebox-1 | PermissionError: [Errno 13] Permission denied: '/app/archivebox/core/migrations/0021_auto_20220724_1200.py'
This happens even if I set up a brand new directory with the :dev branch to create a new database and do setup/init.
If I create the container and spawn a shell to poke around it appears /app has permissions root:root which should be fine, but I guess ArchiveBox writes out the migration files to the /app directory, what's interesting is that this happens even on a totally fresh install.
Here are the permissions:
$ id
uid=999(archivebox) gid=999(archivebox) groups=999(archivebox),29(audio),44(video)
$ ls -lahS /
total 84K
drwxr-xr-x 1 root root 4.0K Jul 24 11:59 .
drwxr-xr-x 1 root root 4.0K Jul 24 11:59 ..
drwxr-xr-x 1 root root 4.0K Jun 9 08:09 app
drwxr-xr-x 1 root root 4.0K Jun 9 00:22 bin
drwxr-xr-x 2 root root 4.0K Mar 19 13:46 boot
drwxr-xr-x 3 archivebox archivebox 4.0K Jul 24 11:59 data
drwxr-xr-x 1 root root 4.0K Jul 24 11:59 etc
drwxr-xr-x 1 root root 4.0K Jun 9 00:00 home
drwxr-xr-x 1 root root 4.0K Jun 9 00:02 lib
drwxr-xr-x 2 root root 4.0K May 27 00:00 media
drwxr-xr-x 2 root root 4.0K May 27 00:00 mnt
drwxr-xr-x 1 root root 4.0K Jun 9 00:16 node
drwxr-xr-x 2 root root 4.0K May 27 00:00 opt
drwx------ 1 root root 4.0K Jun 9 00:21 root
drwxr-xr-x 3 root root 4.0K May 27 00:00 run
drwxr-xr-x 1 root root 4.0K Jun 9 00:03 sbin
drwxr-xr-x 2 root root 4.0K May 27 00:00 srv
drwxrwxrwt 1 root root 4.0K Jun 9 00:30 tmp
drwxr-xr-x 1 root root 4.0K May 27 00:00 usr
drwxr-xr-x 1 root root 4.0K May 27 00:00 var
drwxr-xr-x 5 root root 4.0K Jun 9 00:20 venv
drwxr-xr-x 5 root root 340 Jul 24 11:59 dev
-rwxr-xr-x 1 root root 0 Jul 24 11:59 .dockerenv
dr-xr-xr-x 135 root root 0 Jul 24 11:59 proc
dr-xr-xr-x 12 root root 0 Jul 24 11:59 sys
Update:
May not be a permissions issue, the file doesn't seem to exist
$ cat /app/archivebox/core/migrations/0021_auto_20220724_1159.py
cat: /app/archivebox/core/migrations/0021_auto_20220724_1159.py: No such file or directory
Not too sure what causes this, I looked at the migrations subdir and they're all on git anyways, so I'm guessing it's trying to execute a migration that doesn't exist yet? Maybe @pirate pushed a new build expecting a migration there when there isn't one yet?
Update: Yeah it's a permissions issue, giving archivebox:archivebox ownership of /app/archivebox/core/migrations works:
$ archivebox server --quick-init 0.0.0.0:8000
[i] [2022-07-24 12:54:06] ArchiveBox v0.6.3: archivebox server --quick-init 0.0.0.0:8000
> /data
[^] Verifying and updating existing ArchiveBox collection to v0.6.3...
----------------------------------------------------------------------
[*] Verifying archive folder structure...
+ ./archive, ./sources, ./logs...
+ ./ArchiveBox.conf...
[*] Verifying main SQL index and running any migrations needed...
Operations to perform:
Apply all migrations: admin, auth, contenttypes, core, sessions
Running migrations:
Applying core.0021_auto_20220724_1254... OK
√ ./index.sqlite3
[*] Checking links from indexes and archive folders (safe to Ctrl+C)...
√ Loaded 4 links from existing main index.
> Skipping full snapshot directory check (quick mode)
----------------------------------------------------------------------
[√] Done. Verified and updated the existing ArchiveBox collection.
Hint: To view your archive index, run:
archivebox server # then visit http://127.0.0.1:8000
To add new links, you can run:
archivebox add < ~/some/path/to/list_of_links.txt
For more usage and examples, run:
archivebox help