Skip to content

Partition and format HFS/FAT volumes in the Web UI + SMB install#946

Merged
rdmark merged 51 commits intodevelopfrom
rdmark-webui-image-format
Nov 1, 2022
Merged

Partition and format HFS/FAT volumes in the Web UI + SMB install#946
rdmark merged 51 commits intodevelopfrom
rdmark-webui-image-format

Conversation

@rdmark
Copy link
Copy Markdown
Member

@rdmark rdmark commented Oct 27, 2022

  • New "format as" option when creating new images; removing the image creation options from easyinstall
  • Bring in HFSer as new submodule providing the driver binaries; removing the Lido driver binary from this repo
  • Add SpeedTools driver option
  • Point to github mirror of hfdisk, since the original git server is down
  • While rearranging the easyinstall options, moved the CtrlBoard option up to the main section
  • Add an easyinstall script to configure Samba, while consolidating file sharing with Netatalk

@rdmark rdmark requested a review from nucleogenic October 27, 2022 04:37
@rdmark
Copy link
Copy Markdown
Member Author

rdmark commented Oct 27, 2022

Left to do:

  • Better messages to notify success in the web app
  • Add tests
  • Update scripts that depend on the order of easyinstall options

@rdmark rdmark marked this pull request as ready for review October 27, 2022 16:26
@rdmark
Copy link
Copy Markdown
Member Author

rdmark commented Oct 27, 2022

Some thoughts:

  • Is there a user need for non-Mac file systems and drivers? Maybe use dosfstools for FAT/FAT32/NTFS?
  • It would be cleaner to use Python bytes instead of dd in the shell, but it might be overkill for such a small use case
  • Do we want to make hfdisk and HFSer proper git submodules? It adds a bit of complexity to repo so I'm torn.

Copy link
Copy Markdown
Member

@nucleogenic nucleogenic left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice feature! Just had a quick scan, will do a proper test as soon as I get a chance.

easyinstall.sh Outdated
function cloneHfser() {
if [ ! -f "$BASE/HFSer" ]; then
cd "$BASE" || exit 1
git clone https://github.com/erichelgeson/HFSer.git
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps we could ask @erichelgeson to tag a release? We can't reasonably expect the owner of the repository to maintain backward compatibility with all releases of RaSCSI indefinitely.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since you are just taking the drivers out, please just copy/host the drivers somewhere. I will be archiving this repo as it's little more than a python wrapper for dd and I wont be updating it.

@nucleogenic
Copy link
Copy Markdown
Member

  • Is there a user need for non-Mac file systems and drivers? Maybe use dosfstools for FAT/FAT32/NTFS?

Possible YAGNI, but I am by no means the best person to answer this.

@rdmark rdmark requested a review from nucleogenic October 27, 2022 22:49
@rdmark rdmark changed the title Partition and format HFS volumes in the Web UI Partition and format HFS volumes in the Web UI + SMB install Oct 28, 2022
@rdmark
Copy link
Copy Markdown
Member Author

rdmark commented Oct 28, 2022

@nucleogenic Sorry for the feature creep, but I also added an SMB installation script (since I was already touching the sequence of those options.) This is something that has been requested by the community in the past and I just felt a bit inspired to make it happen.

@rdmark
Copy link
Copy Markdown
Member Author

rdmark commented Oct 29, 2022

@uweseimet As an Atari ST user, I wanted to get your opinion on the ability of mkfs.fat to create an Atari compatible FAT file system. In this PR, I have the Web Interface run mkfs.fat against newly created images to assist in creating file systems on the fly, to assist in bootstrapping a usable volume.

Do you think this is useful for Atari ST users, or is it redundant?

The command looks something like:

mkfs.fat -A -n LABEL path/to/image

@uweseimet
Copy link
Copy Markdown
Contributor

uweseimet commented Oct 30, 2022

@rdmark That's hard to tell. Atari users usually have more than one partition (i.e. filesystem) on a drive. If you want to boot from a drive, a hard disk driver has to be installed on it. The hard disk driver software also includes a tool to partition the drive, i.e. to create several filesystems on it, which are then assigned the drive letters C:, D: etc. The resulting layout is very similar to what you have with Windows. Depending on the OS version the maximum FAT16 partition size the Atari supports is 1 GB. This means that in order to use the full capacity of your medium you need several partitions.
In terms of Linux commands Atari users would first use fdisk to create as many partitions as they need, and then they would run mkfs for each partition in order to create an Atari compatible filesystem on it. Atari hard disk driver tools do all this in a single step.
There is software for the Atari which supports FAT32 or Linux EXT2 partitions. This is why sometimes you do not just want to create FAT16 partitions but also others.
All in all, just being able to create a single partition on an image is probably not that useful for Atari users.
In addition, I think just running mkfs on an image file will not create a partition table (like fdisk does) but just a single filesystem (super floppy layout). Most hard disk drivers for the Atari do not support this, and you would not be able to boot from such a filesystem.

echo "Lido driver couldn't be found. Make sure RaSCSI is up-to-date with git pull"
return 1
fi
sudo cp hfdisk "$HFDISK_BIN"
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Worth cleaning up the extracted hfdisk dir as well as the archive?

@rdmark
Copy link
Copy Markdown
Member Author

rdmark commented Oct 31, 2022

@uweseimet Thanks for breaking down the requirements for a usable disk drive partition and file system on Atari ST. I can see how a simplistic option like this won't be helpful for Atari ST users, so I'll probably remove it for now.

In my initial reading of the mkfs.fat manpage I got the impression that it creates a singular partition on the drive if one does not exist, but I haven't been able to confirm that the resulting image leads to a usable drive image even in the DOS/Windows world. The system sees the MBR, but no usable volume with drive letter. You may be correct that fdisk will be required even for this case.

@uweseimet
Copy link
Copy Markdown
Contributor

uweseimet commented Oct 31, 2022

@rdmark Regarding drives without MBR (super floppy layout) there definitely are DOS versions (maybe not Windows) that support it, just as my hard disk software for the Atari does. But in practice this format at least for the Atari does not have any relevance. I mainly support it because I can and because it exists. There might even be somebody using it, you never know ;-).

@rdmark rdmark marked this pull request as draft October 31, 2022 20:00
@rdmark rdmark marked this pull request as ready for review November 1, 2022 04:46
@rdmark rdmark requested a review from nucleogenic November 1, 2022 04:46
@rdmark
Copy link
Copy Markdown
Member Author

rdmark commented Nov 1, 2022

@nucleogenic This PR is stabilized now. I have tested that the resulting FAT volumes are now recognized out of the box on a WinXP system.


try:
process = run(
["kpartx", "-av", full_file_path],
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I get TypeError: sequence item 2: expected str instance, PosixPath found on Python 3.7; it's fixed when full_file_path is cast as str.



# route("/files/create", methods=["POST"])
def test_create_file_and_format_fat(http_client, list_files, delete_file):
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This test fails for me because the device mapper isn't currently available in our Docker images.

We should skip this test for now with:

def test_create_file_and_format_fat(env, http_client, list_files, delete_file):
    if env["is_docker"]:
        pytest.skip("Test not supported in Docker environment.")

@rdmark rdmark merged commit 85edd50 into develop Nov 1, 2022
@rdmark rdmark deleted the rdmark-webui-image-format branch November 1, 2022 23:43
@sonarqubecloud
Copy link
Copy Markdown

sonarqubecloud bot commented Nov 2, 2022

SonarCloud Quality Gate failed.    Quality Gate failed

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 0 Code Smells

0.0% 0.0% Coverage
0.0% 0.0% Duplication

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants