Edit: they both had a cache miss
GHA took 25 min while CircleCi took 23

Cached took:
GHA: 6 min
CircleCI: 3 min

Now they are both at 4 min

Do the docs & translations need to run on each commit? Can i make a nightly / twice a day GHA in the doc repo to check if master changed and if yes rebuild the docs?

Changing translations from nightly to every commit on master was a great improvement imho, and I'm extremely satisfied about changing it, so definitely a yes on that one.

For the docs it probably matters less, but a discrepancy is still a bit annoying and would actively get in the way of my current workflow. Most contributors don't run ldoc locally first, and I rarely fetch their branch just to run ldoc; it's easier to check the results 3 minutes later and usually it's perfectly fine. This would be impossible if you made it worse, though I could live with it. But it only takes a handful of seconds so I'm not sure what the big deal is anyway? ;-)

Do the docs & translations need to run on each commit? Can i make a nightly / twice a day GHA in the doc repo to check if master changed and if yes rebuild the docs?

Changing translations from nightly to every commit on master was a great improvement imho, and I'm extremely satisfied about changing it, so definitely a yes on that one.

For the docs it probably matters less, but a discrepancy is still a bit annoying and would actively get in the way of my current workflow. Most contributors don't run ldoc locally first, and I rarely fetch their branch just to run ldoc; it's easier to check the results 3 minutes later and usually it's perfectly fine. This would be impossible if you made it worse, though I could live with it. But it only takes a handful of seconds so I'm not sure what the big deal is anyway? ;-)

np, it is just annoying to trigger a commit in a different repo with GHA

Within an organization even? That's a bit annoying.

Within an organization even? That's a bit annoying.

Yes :(
The simplest way is to create a Personal Access Token with the public_repo scope and set this token as a organization secret that can be used in the CI.
This PAT can only be created by a user and has that users rights. So i am not going to make a personal PAT and put it in the org. If someone else wants to volunteer is good. if not i was thinking of making a new GH account for just this, invite it with read only into the org and use it's PAT for the CI.

The simplest way is to create a Personal Access Token with the public_repo scope and set this token as a organization secret that can be used in the CI.j

It's a bit surprising that this is also necessary within GitHub, but that's 100% identical to the former Travis and the current CircleCI setup.

So i am not going to make a personal PAT and put it in the org. If someone else wants to volunteer is good.

You mean as a security precaution? Unless GHA has some serious design flaws a token is once entered and never shown to anyone. The only available option afterwards is deleting them. I'm fine with generating new ones and entering them in the secrets here.

So i am not going to make a personal PAT and put it in the org. If someone else wants to volunteer is good.

You mean as a security precaution? Unless GHA has some serious design flaws a token is once entered and never shown to anyone. The only available option afterwards is deleting them. I'm fine with generating new ones and entering them in the secrets here.

If anyone merges a malicious or buggy GHA they can exfiltrate the PAT or just commit etc as me in any of my repos. This is why github by default won't run workflows for first time contributors as a precaution in case they change or take advantage of a gha.

Looks like it should be able to live next to CircleCI now? If so I'm fine with merging this already to start testing the basics in the wild. ^_^

👍
just maybe give it a day for my comments below.
Part of this PR is that the gha does not not depend on scripts in the .ci/ folder. all the functionality is achieved through use of ./kodev. This allows us to run the same tests and lint locally.

Do we still use travis anywhere? if not can i remove its third party permissions for the org?

You can remove it. Travis.org stopped working some 4-5 months ago, cf. koreader/crengine#451

@yparitcher

If anyone merges a malicious or buggy GHA they can exfiltrate the PAT or just commit etc as me in any of my repos. This is why github by default won't run workflows for first time contributors as a precaution in case they change or take advantage of a gha.

Travis could just run while not passing any secrets. (I.e., $SECRET_PAT would just be empty.)

This is what recently developed into an issue with CircleCI, where it refused to run at all, except without GHA's nice "authorize" (except I don't want to authorize anything at all obviously!).

If that's a problem, I would be more inclined to cancel GHA completely and just do some GHA stuff from the docs & translation repos once an hour or so.

I.e., removing secrets from CircleCI will fix all CircleCI issues. If GHA has the same issues it doesn't fix anything.

Merging the disscussion from above and koreader/koreader-base#1442

the primary issue with CircleCI ... show the output.[1]
[1] That's new as well. If I'm not mistaken it always used to show it to you regardless if you were logged in.

This is fixed for me now that we have CircleCi checks installed.

This is what recently developed into an issue with CircleCI, where it refused to run at all, except without GHA's nice "authorize" (except I don't want to authorize anything at all obviously!).

I am not sure what "authorize" you are referring to. Is this only because of the docs/translations?

also as i asked before in #8304 (comment)

github action are run with a GITHUB_TOKEN that only has permissions in the current repo.
In order to commit in a different repo on can:
make a user PAT, however the drawback is that if the person leaves the org it will stop working, and there are no ganular permissions, the PAT an access any repo of the user.

With a organization Github Apps one can give the app only repo permissions and these permissions will only be applied to the repos the app is installed to.
we can make a "commit" Github App and install it in the doc and translation repos. this can then be used by a workflow in the koreader repo to commit with the auth of the app. For details see this blog post

Can we change the default permission for the GITHUB_TOKEN to read only as i don't think we are using github actions anywhere to write

I am not sure what "authorize" you are referring to. Is this only because of the docs/translations?

Yes. There are (or were) three issues:

1. Sometimes PRs won't run the checks (and worse, they won't show this as a failed check). That's because of this:
   

   koreader/.circleci/config.yml

   Lines 77 to 82 in 6c9e222

   	workflows:
   	version: 2
   	build:
   	jobs:
   	- build:
   	context: koreader-vars

   
   
   Those are personal access tokens I created, used only in CircleCI. Travis and I'd swear also CircleCI used to just run the PR without populating the environment variables. That was ideal. Their focus is on private organizations who actually pay for the service, so it doesn't seem to have received much attention.
2. Since fairly recently (insofar as 2019 or so is recent), CircleCI has started requiring you log in to see the results. But apparently this has recently ceased to be an issue, in the actual definition of recently since I know I had to log in just weeks ago.
   
3. The newer Docker image segfaulted. I was hoping that GHA might help, but it did far worse than just not helping. Even getting to a point where it'd segfault proved to be an ordeal.

Given the three above facts, a year ago GHA seemed like the most logical course of action. With 2 and 3 taken out of the equation there's no need to switch to GHA wholesale.

There's may still be a role for GHA to play to work around point 1, and I'm not opposed to GHA taking on everything (in front), but especially 2 and to a lesser extent 3 were always the primary reason to want to move over to GHA. Because 2 would be fixed for sure and 3 might be fixed as a side effect.

Can we change the default permission for the GITHUB_TOKEN to read only as i don't think we are using github actions anywhere to write

Wouldn't adding release builds be write?

I am not sure what "authorize" you are referring to. Is this only because of the docs/translations?

I assume that without any secrets, you wouldn't have to click the "run this action" or whatever it says exactly button? That's what I mean. I don't want random PRs to have any secrets, like in Travis. If PRs get secrets, then it'd definitely have to be a dedicated user or some sort.

Since fairly recently (insofar as 2019 or so is recent), CircleCI has started requiring you log in to see the results. But apparently this has recently ceased to be an issue, in the actual definition of recently since I know I had to log in just weeks ago.

Ohhh and point 1 also somewhat ceased to be an issue, at least in the sense that it now clearly says unauthorized. (See over on #8604 right now.)

That means that every single reason I had to avoid spending time on CircleCI and to migrate to GHA or possibly GitLab CI instead is now gone. [Edit: which doesn't mean we should throw away this PR!]

There is still the issue that we don't have CircleCI running at all (not even showing as "unauthorized") on PRs from @hius07 and @zwim ...
(All these past and current headaches really have me wanting to get rid of CircleCI :)

I've had much worse headaches with GHA already unfortunately. :-)

There is still the issue that we don't have CircleCI running at all (not even showing as "unauthorized") on PRs from @hius07 and @zwim ...

I'll fix that. I was kept from it by the fact that if it failed it wasn't a big red x and point 2.

I added the && git submodule --quiet foreach to do the submodules. That command only deals with the repo in question:

$ git grep -lE '^#!(/usr)?/bin/(env )?(bash|sh)'
.ci/after_success.sh
.ci/before_install.sh
.ci/build.sh
.ci/check.sh
.ci/common.sh
.ci/fetch.sh
.ci/helper_luarocks.sh
.ci/helper_shellchecks.sh
.ci/install.sh
.ci/script.sh
.ci/test.sh
kodev
platform/appimage/AppRun
platform/cervantes/disable-usbms.sh
platform/cervantes/disable-wifi.sh
platform/cervantes/enable-usbms.sh
platform/cervantes/enable-wifi.sh
platform/cervantes/koreader-standalone.sh
platform/cervantes/koreader.sh
platform/cervantes/obtain-ip.sh
platform/cervantes/release-ip.sh
platform/cervantes/restore-wifi-async.sh
platform/cervantes/spinning_zsync
platform/common/spinning_zsync
platform/debian/do_debian_package.sh
platform/debian/koreader.sh
platform/kindle/extensions/koreader/bin/koreader-ext.sh
platform/kindle/koreader.sh
platform/kindle/libkohelper.sh
platform/kobo/disable-wifi.sh
platform/kobo/enable-wifi.sh
platform/kobo/fmon/koreader.sh
platform/kobo/koreader.sh
platform/kobo/nickel.sh
platform/kobo/obtain-ip.sh
platform/kobo/release-ip.sh
platform/kobo/restore-wifi-async.sh
platform/mac/do_mac_bundle.sh
platform/pocketbook/koreader.app
platform/pocketbook/system_koreader.app
platform/remarkable/disable-wifi.sh
platform/remarkable/enable-wifi.sh
platform/remarkable/koreader.sh
platform/sony-prstux/resume.sh
platform/sony-prstux/set-wifi.sh
platform/sony-prstux/suspend.sh
platform/ubuntu-touch/koreader.sh
plugins/backgroundrunner.koplugin/luawrapper.sh
plugins/backgroundrunner.koplugin/wrapper.sh

However, in modern Git you can also do git grep --recurse-submodules -lE '^#!(/usr)?/bin/(env )?(bash|sh)'. Your results suggest there's a way to enable that through a setting as well, and either you did it and forgot or your distro did it for you.

Edit: that is, back in Git 2.71.9.1 or something it either wasn't an option or I wasn't aware of it, and you had to use that git submodule foreach construct.

The issue with the context suffered by @hius07, @zwim and others should be resolved by ec280f8.

That might look something like this:

Codecov Report

Merging #8607 (ec280f8) into master (ec280f8) will not change coverage.
The diff coverage is n/a.

❗ Current head ec280f8 differs from pull request most recent head c8b94bd. Consider uploading reports for the commit c8b94bd to get more accurate results

@@           Coverage Diff           @@
##           master    #8607   +/-   ##
=======================================
  Coverage   39.67%   39.67%           
=======================================
  Files         289      289           
  Lines       50259    50259           
=======================================
  Hits        19941    19941           
  Misses      30318    30318           

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update ec280f8...c8b94bd. Read the comment docs.

Is this PR still relevant?

Doing something like that kodev-check() would be better regardless so if it doesn't bother you too much I'd prefer to leave it open for the moment (but just so it's easy to find as documentation).