Skip to content

chore(arch): change repo architecture#153

Merged
nikoksr merged 45 commits intomasterfrom
chore-change-repo-architecture
Jul 27, 2020
Merged

chore(arch): change repo architecture#153
nikoksr merged 45 commits intomasterfrom
chore-change-repo-architecture

Conversation

@nikoksr
Copy link
Copy Markdown
Collaborator

@nikoksr nikoksr commented Jul 27, 2020

Summary

Introduce a cleaner repo architecture which prevents import cycles.

Proposed Changes

  • Split up interfaces and implementations into separate packages
  • Add a service and store for every domain
  • Move purely internal packages (packages that are not meant to be imported by others) into the internal/ directory

nikoksr added 30 commits July 12, 2020 19:55
Because of the previous project architecture we had many problems with
include cycles when dealing with models and remote repositories. The
necessary modifications led to weird looking function signatures and
calls.
Project has now a clean and easier to understand architecture. In case
of the previously called models, I split the type declarations from the
implementations. Every major domain is now described in pkg/domain like
for example package and project. pkg/domain declares the type and its
store and service respectively. The store interface describes the way a
domain talks to a storage provider like a database. The service
interface describes how the implementer would usually interact with the
domain. Some service functions use store functions internally.
Instead of exporting a big and descriptive example package config, -t
now exports a minimal package config which serves as a quick entrypoint
for the creation of a new package. The big descriptive config file now
lives in the github repo and will serve as a reference for new users.
A stricter drop-in replacement for gofmt and goimports.
Configs are no longer downloaded but rather generated and thus the
deploy command no longers depends on projis current version.
The main config now holds a field which lets the user specify a regex
which is used with the 'package import' command. The specified regex is
used to exclude folders and files from imports via dir/repo-structure
and collections. In case that the '-e/--exclude' flag of said command
gets passed, it overwrites the regex specified in the config file.
The statuswriter package is meant to print the satuses of mulitple
actions to the cli concurrently. The statuswriter can hold multiple
sinks. Each sink will write and update a single line in the terminal.
Its main use case currently lies in giving good feedback to the use
while importing multiple packages. Each packages import status will be
written in a dedicated line. The line will be rewritten when the state
of the package changes. E.g. the line will update when the status
swiches from importing the package to storing it.
Exlcude now supports regex patterns. Default value is set in proji's
main config under import.exclude. This value is overwriteable by the
optional exclude flag.
Package imports are now a lot more verbose. Using the statuswriter
package we can now print the import statuses of multiple packages
concurrently.
loadConfig now loads flags too. In example the exclude flag.
Import.exclude may be defined via the config or command flag. The flag
should overwrite the config value which is done in loadConfig.
Configs now go along with the latest internal config changes and the
scripts were ported to lua.
Records will no longer be soft-deleted, meaning that a for example
deleted package configuration will no longer be recoverable. This
feature introduced more difficulties than it solved. Soft-deletes might
return in the future but for now I want a stable core for the storage
system and not fight with duplicate soft-deleted records.
nikoksr added 11 commits July 23, 2020 16:41
Via the function parameter loadDependencies you can now decide whether
package dependencies should be loaded or not. One use case is the
'package ls' command which only displays the available packages with
their names and labels. Until now this call loaded all packages with
their respective dependencies. After this change the call will only load
the packages name, label and description and ignore its dependencies.
For small packages the performance that I got out of gorm was very good.
Please consider that at the time writing this gorm 2.0 is just on the
edge of being released and its documentation is not yet at 100%.
In the extreme case of importing something like the structure of the
GitHub repository torvalds/linux gorm showed significant performance
issues. Importing a package of that size with gorm took literally
minutes. And even deleting something of that size and complexity took a
noticeable time.
I replaced almost all storage interactions that are performance critical
with raw sql and importing torvalds/linux now only takes about 5 seconds
on my laptop which is good enough for now. 99% of packages will be a
fraction of that size and complexity.
Query building is now done by the sql library and not manually.
Previously we were using filters to filter the imported paths and
packages. Filter was meant to allow you to say something like 'take only
this package and that package' or 'take everything except this'. Regexp
from the go stdlib guarantees a consistent runtime for all its functions
and thus does not include support for negative lookaheads which would be
needed for 'only-this' filters. Thats why we now will explicitly use the
term excludes to emphasise that the regex is meant to exlude paths and
or packages.
Query that were set together out of a base and conditions part were
missing a space to separate them. Led to panics.
@codecov-commenter
Copy link
Copy Markdown

Codecov Report

Merging #153 into master will decrease coverage by 6.06%.
The diff coverage is 1.20%.

Impacted file tree graph

@@            Coverage Diff            @@
##           master    #153      +/-   ##
=========================================
- Coverage    7.68%   1.61%   -6.07%     
=========================================
  Files          36      44       +8     
  Lines        1704    1915     +211     
=========================================
- Hits          131      31     -100     
- Misses       1563    1882     +319     
+ Partials       10       2       -8     
Impacted Files Coverage Δ
cmd/completion.go 0.00% <0.00%> (ø)
cmd/init.go 0.00% <0.00%> (ø)
cmd/package.go 0.00% <0.00%> (ø)
cmd/packageAdd.go 0.00% <0.00%> (ø)
cmd/packageExport.go 0.00% <0.00%> (ø)
cmd/packageImport.go 0.00% <0.00%> (ø)
cmd/packageLs.go 0.00% <0.00%> (ø)
cmd/packageRm.go 0.00% <0.00%> (ø)
cmd/packageShow.go 0.00% <0.00%> (ø)
cmd/projectAdd.go 0.00% <0.00%> (ø)
... and 57 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 6cd70da...9b2915e. Read the comment docs.

@nikoksr nikoksr merged commit 7955696 into master Jul 27, 2020
@nikoksr nikoksr deleted the chore-change-repo-architecture branch July 27, 2020 22:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants