chore(arch): change repo architecture#153
Merged
Conversation
Because of the previous project architecture we had many problems with include cycles when dealing with models and remote repositories. The necessary modifications led to weird looking function signatures and calls. Project has now a clean and easier to understand architecture. In case of the previously called models, I split the type declarations from the implementations. Every major domain is now described in pkg/domain like for example package and project. pkg/domain declares the type and its store and service respectively. The store interface describes the way a domain talks to a storage provider like a database. The service interface describes how the implementer would usually interact with the domain. Some service functions use store functions internally.
Instead of exporting a big and descriptive example package config, -t now exports a minimal package config which serves as a quick entrypoint for the creation of a new package. The big descriptive config file now lives in the github repo and will serve as a reference for new users.
A stricter drop-in replacement for gofmt and goimports.
Configs are no longer downloaded but rather generated and thus the deploy command no longers depends on projis current version.
The main config now holds a field which lets the user specify a regex which is used with the 'package import' command. The specified regex is used to exclude folders and files from imports via dir/repo-structure and collections. In case that the '-e/--exclude' flag of said command gets passed, it overwrites the regex specified in the config file.
The statuswriter package is meant to print the satuses of mulitple actions to the cli concurrently. The statuswriter can hold multiple sinks. Each sink will write and update a single line in the terminal. Its main use case currently lies in giving good feedback to the use while importing multiple packages. Each packages import status will be written in a dedicated line. The line will be rewritten when the state of the package changes. E.g. the line will update when the status swiches from importing the package to storing it.
Exlcude now supports regex patterns. Default value is set in proji's main config under import.exclude. This value is overwriteable by the optional exclude flag. Package imports are now a lot more verbose. Using the statuswriter package we can now print the import statuses of multiple packages concurrently.
loadConfig now loads flags too. In example the exclude flag. Import.exclude may be defined via the config or command flag. The flag should overwrite the config value which is done in loadConfig.
Configs now go along with the latest internal config changes and the scripts were ported to lua.
Records will no longer be soft-deleted, meaning that a for example deleted package configuration will no longer be recoverable. This feature introduced more difficulties than it solved. Soft-deletes might return in the future but for now I want a stable core for the storage system and not fight with duplicate soft-deleted records.
Via the function parameter loadDependencies you can now decide whether package dependencies should be loaded or not. One use case is the 'package ls' command which only displays the available packages with their names and labels. Until now this call loaded all packages with their respective dependencies. After this change the call will only load the packages name, label and description and ignore its dependencies.
For small packages the performance that I got out of gorm was very good. Please consider that at the time writing this gorm 2.0 is just on the edge of being released and its documentation is not yet at 100%. In the extreme case of importing something like the structure of the GitHub repository torvalds/linux gorm showed significant performance issues. Importing a package of that size with gorm took literally minutes. And even deleting something of that size and complexity took a noticeable time. I replaced almost all storage interactions that are performance critical with raw sql and importing torvalds/linux now only takes about 5 seconds on my laptop which is good enough for now. 99% of packages will be a fraction of that size and complexity.
Query building is now done by the sql library and not manually.
Previously we were using filters to filter the imported paths and packages. Filter was meant to allow you to say something like 'take only this package and that package' or 'take everything except this'. Regexp from the go stdlib guarantees a consistent runtime for all its functions and thus does not include support for negative lookaheads which would be needed for 'only-this' filters. Thats why we now will explicitly use the term excludes to emphasise that the regex is meant to exlude paths and or packages.
Query that were set together out of a base and conditions part were missing a space to separate them. Led to panics.
Codecov Report
@@ Coverage Diff @@
## master #153 +/- ##
=========================================
- Coverage 7.68% 1.61% -6.07%
=========================================
Files 36 44 +8
Lines 1704 1915 +211
=========================================
- Hits 131 31 -100
- Misses 1563 1882 +319
+ Partials 10 2 -8
Continue to review full report at Codecov.
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Introduce a cleaner repo architecture which prevents import cycles.
Proposed Changes
internal/directory