Skip to content

0.9.1#644

Merged
ca333 merged 30 commits intomasterfrom
dev
Jul 18, 2025
Merged

0.9.1#644
ca333 merged 30 commits intomasterfrom
dev

Conversation

@ca333
Copy link
Copy Markdown

@ca333 ca333 commented Mar 14, 2025

No description provided.

DeckerSU and others added 15 commits September 30, 2024 09:27
download-artifact@v4+ is not currently supported on GHES yet.
* cc: fix ed25519 signatures malleability

- #630
- https://soatok.blog/2024/08/14/security-issues-in-matrixs-olm-library/#vuln-ed25519

Actually, the current CC code doesn’t use Ed25519 signatures, so `CVE-2024-45193` has no impact on Komodo (KMD) or any existing assetchains. However, since CC could potentially use these types of signatures in the future (e.g., for newly developed CCs), we’ve added a `0 <= s < L` check to prevent signature malleability.

* add ed25519 signature malleability test

* use int instead of size_t in 0 <= s < L check loop

using a signed integer type (int) is preferable here,
to avoid potential issues with unsigned underflow.

* cc: test, update pytest ver. requirement

addressed in #631
…u.org (#635)

As of August 30, 2024, `git.savannah.gnu.org` returned a
502 error when trying to download `config.guess` and `config.sub`.
For example, attempting to download [this link](https://git.savannah.gnu.org/gitweb/?p=config.git;a=blob_plain;f=config.sub;hb=HEAD)
returned a 502 error. As a result, all scripts, workflows, etc.,
that relied on getting fresh versions of `config.{sub,guess}`
from `git.savannah.gnu.org` are failing. To prevent this, we
disabled updating to the fresh versions of these scripts.

- DeckerSU/KomodoOcean@4a29828
- #633
* GLEEC_OLD: exception to fix following the notarizations

This configuration assumes that we have the `~/.komodo/GLEEC_OLD`
directory for the old GLEEC chain. The daemon should be launched
with `-datadir=$HOME/.komodo/GLEEC_OLD`, and the data directory
should contain `GLEEC.conf` along with a symlink to it
named `GLEEC_OLD.conf`.

* timeshifting transient verb

---------

Co-authored-by: smk762 <smk@komodoplatform.com>
this will help Iguana identify the change UTXOs and use them for splitting.
Artifact actions v3 will be deprecated by December 5, 2024.
* build: boost, prevent use of std::unary_function

For Boost 1.72 we should use -D_HAS_AUTO_PTR_ETC=0, for newer versions
-DBOOST_NO_CXX98_FUNCTION_BASE.

- boostorg/container_hash@fd310d2
- boostorg/functional@6a573e4
- bitcoin/bitcoin@880d4aa
- bitcoin/bitcoin@d4c59da
- DeckerSU/KomodoOcean@0742048

This will remove warnings like the one shown below during build with GCC 13:

```
/home/decker/komodo/depends/x86_64-unknown-linux-gnu/share/../include/boost/container_hash/hash.hpp:130:33: warning: ‘template<class _Arg, class _Result> struct std::unary_function’ is deprecated [-Wdeprecated-declarations]
  130 |         struct hash_base : std::unary_function<T, std::size_t> {};
      |                                 ^~~~~~~~~~~~~~
In file included from /usr/include/c++/13/string:49,
                 from /usr/include/c++/13/bits/locale_classes.h:40,
                 from /usr/include/c++/13/bits/ios_base.h:41,
                 from /usr/include/c++/13/ios:44,
                 from /usr/include/c++/13/ostream:40,
                 from /home/decker/komodo/depends/x86_64-unknown-linux-gnu/share/../include/gtest/gtest.h:55,
                 from test-komodo/test_legacy_events.cpp:1:
/usr/include/c++/13/bits/stl_function.h:117:12: note: declared here
  117 |     struct unary_function
      |            ^~~~~~~~~~~~~~
```

* build: add BOOST_CPPFLAGS display for configure
Been a few years since we last updated these.

- bitcoin/bitcoin#27508
- #636
- DeckerSU/KomodoOcean#84

Taking into account that the `x86_64-unknown-linux-gnu` triplet
has been changed to `x86_64-pc-linux-gnu` with the new files.
* introduce DEFAULT_TXINDEX (true)

as a part of constrain constant values to a single location in code

* allow to start without komodo.conf for KMD

No komodo.conf file is OK
The Ubuntu 20.04 runner image will be fully unsupported by April 1, 2025.
ci/cd: update runner image ubuntu-20.04 -> ubuntu-22.04
@ca333 ca333 marked this pull request as draft March 14, 2025 15:14
@DeckerSU
Copy link
Copy Markdown

I will first add a few important changes to komodod unit tests in the dev branch, as it turned out that the success of some tests depends on their execution order. For example, a test sequence of 1,2,3 might complete successfully, but 1,3,2 could fail.

This behavior is caused by some global variables that get corrupted during certain tests. I have already identified these issues and fixed most of them in the KomodoOcean repo. However, I strongly want to include these fixed tests in this release.

The estimated completion time is by Monday, or possibly even earlier.

@ca333
Copy link
Copy Markdown
Author

ca333 commented Mar 15, 2025

I will first add a few important changes to komodod unit tests in the dev branch, as it turned out that the success of some tests depends on their execution order. For example, a test sequence of 1,2,3 might complete successfully, but 1,3,2 could fail.

This behavior is caused by some global variables that get corrupted during certain tests. I have already identified these issues and fixed most of them in the KomodoOcean repo. However, I strongly want to include these fixed tests in this release.

The estimated completion time is by Monday, or possibly even earlier.

thanks for the efforts!

@DeckerSU
Copy link
Copy Markdown

DeckerSU commented Mar 15, 2025

thanks for the efforts!

In order to publish the release, we need to merge at least two PRs:

These PRs don’t have reviewers yet, but I will assign them shortly. Before merging anything into main, we must have these changes in dev first.

Additionally, the unit tests PR should be tested by the QA team, following this small instruction. Of course, I have already tested it myself, but having a second pair of eyes would definitely be beneficial.

@DeckerSU
Copy link
Copy Markdown

I also created a PR to update the key information in README.mdPR #647. If you have any other suggestions for updates or changes, feel free to add commits to the patch-readme-md-2005 branch.

```
/usr/bin/ld: libcc.a(libcc_a-cclib.o): relocation R_X86_64_32 against `.rodata.str1.8' can not be used when making a shared object; recompile with -fPIC
libcc.a: error adding symbols: Bad value
collect2: error: ld returned 1 exit status
Makefile:2305: recipe for target 'komodod' failed
make[2]: *** [komodod] Error 1
```
* ci/cd: switch to ubuntu-22.04 for windows build

We don't want the build to fail with the next `ubuntu-latest`
update, so for now, we will stick with `ubuntu-22.04`,
which is working perfectly.

* cd: make Windows Build (mingw) step in CD the same as in CI

* add clean-help-dev.sh script, to proper clean build directory

temp solution which purpose is to clean artifacts between build
for various platforms

* clean-help-dev.sh: display echo confirmation after removing build files
…ependent (#646)

* tests, debug: add debug printout for Global Variables

* tests: fix fTxIndex set after exit from GMPArithTests.RewardsTest

DeckerSU/KomodoOcean#95 (comment)

* tests: fix LegacyEventsTests removing temp directory, close komodoevents

Windows specifics: The OS more strictly locks files if they are in use, even if their usage has been completed in the code. We should not leave the komodoevents file descriptor open.

TODO: The **signedmasks** descriptor is also handled during komodod / komodo-test execution and is never closed (!)

* tests: prevent ASSETCHAINS_MAGIC corruption in ParseArgumentsTests

* tests: prevent ASSETCHAINS_CCDISABLES corruption in ParseArgumentsTests

* tests: move deleteIfUsedBefore to testutils.h

* tests: fix TestCoinImport global variables set/unset

* debug: add assetchain output stream operator<<

* tests: disable GLEEC example, as an old GLEEC chain now require -datadir

* tests: fix TestNotary.* tests set, now it's tied to NUM_KMD_SEASONS

* tests: fix global fTxIndex state in TestChain

* tests: fix TestParseNotarisation.FilePaths test

* test: fix consensus params after chainparams_commandline call

* tests: fix TestAlerts.AlertNotify test

* tests: fix TestParseNotarisation.test_prevMoMheight tests

last.MoM could be modified by other tests, so we need a workaround.

* tests: fix TestParseNotarisation tests

IS_KOMODO_NOTARY changes critical for LegacyEvents tests.

* tests: check IS_KOMODO_NOTARY == false condition, for LegacyEventsTests

* tests: fix IsInitialBlockDownload() debug output in LegacyEventsTests

we want to output value, not address ...

* tests: prevent ASSETCHAINS_CC corruption

* tests: prevent ASSETCHAINS_COMMISSION corruption

* tests: add test_block suite comments

* tests: make Mempool tests independent from chainActive state

prevent crash in GetMedianTimePast calculation

* tests: fix TestEvalNotarisation.test_komodo_notarysinit test

* test: fix TestEvalNotarisation.test_komodo_notaries test

* tests: temp fix for TestNotary.KomodoNotaries test

* tests: fix TestEvalNotarisation.test_komodo_notarysinit

static hwmheight can be non-zero after komodo_init(...) after some
other tests ... so we should clear it once again.

* CC: fix CCShowStructure behavior if nullptr is passed

* tests: fix all TestBet.* set (undefined behavior inside komodo_nextheight)

* tests: move GTEST_COUT_COLOR to testutils.h

* tests: add GetNetworkByIdStr function (id_str -> network)

* tests: correct restore old network in Mempool.* and PoW.* tests

* tests: fix GMPArithTests.RewardsTest (should use REGTEST)

* tests: fix TestParseNotarisation.prevMoMheight test

* tests: fix accessing real ~/.komodo directory instead set -datadir

because of path caching ...
@DeckerSU
Copy link
Copy Markdown

DeckerSU commented Mar 23, 2025

@ca333 Seems we’re ready for the release. The only thing left is to review the wording in this PR: #647 and get it merged as well.

After that, we can merge dev into master, and I’ll start preparing the binaries and everything else needed.

p.s. Don't merge please, until I fix CD create_release step / ensure it works correctly.

* [ci/cd] save time on removal and upgrade packages

we should be fine with runner defaults for CI ...

* [ci/cd] switch to ubuntu-22.04 and don't remove packages for windows cd
@DeckerSU
Copy link
Copy Markdown

All CI/CD-related issues have been fixed. However, there are still a few things we should rework. For example, we need to stop using the archived actions/create-release action.

We should also optimize the build flags used for artifacts—most likely, we don't need all the debug information in the Linux komodod binary, which currently makes it around 280 MB.

I'll gradually implement these changes moving forward.

@TheComputerGenie
Copy link
Copy Markdown

TheComputerGenie commented Mar 24, 2025

Most of this code:
https://github.com/KomodoPlatform/komodo/blob/e9e37e1de308f766a3e801bec142b4f1a750f4dc/.github/workflows/komodod_cd.yml#L41-L50
is pointless.

Assuming toolchest is actually needed for test builds, it's not needed for CD. Also, there's no reason to burn time uninstalling things for the CD using 20 that were an issue when GitHub had an issue for 5 days on 16.

At most, CD only requires libcurl4-gnutls-dev and software-properties-common (the latter of which is really only needed for toolchest) on the GitHub runners since all other system dependencies (that aren't in depends build) are preinstalled on the runners.

@DeckerSU
Copy link
Copy Markdown

If by toolchest you meant the ubuntu-toolchain-r/test repository, which was recently removed from the GitHub runner images (as mentioned here: actions/runner-images#9679), then yes — we should probably reconsider this at some point and switch back to using the pre-installed gcc/g++ versions from the runner image.

More broadly, I agree with you — we shouldn't spend time and resources on unnecessary steps. For example, we can remove the Linux CD package cleanup steps, which are likely not needed.

Also, it might be worth considering a move to dockerized builds, like I’ve already done in the KomodoOcean repo. That would help eliminate dependencies on the runner image content entirely. But this would be a point of discussion with the QA team, to ensure compatibility and reliability.

Overall, thank you for your input — your suggestions will definitely be considered for the next release cycle. For this upcoming 0.9.1 release, the goal wasn’t to rework all of CI/CD, but simply to fix what was broken and ensure builds are working again.

Of course, testing and debugging CI/CD in the main repo is quite inconvenient, so I’ll likely handle future improvements in a separate repo and bring them over as a clean PR once everything is working as expected.

@DeckerSU
Copy link
Copy Markdown

p.s. Also, for building the Linux version—if you're running on a clean Ubuntu 22.04 image—you don’t need many extra dependencies beyond:

git build-essential autoconf automake curl libtool pkg-config bsdmainutils

So, for example, you can run:

docker run -it --rm -v "$(pwd)":/app -w /app ubuntu:22.04 bash

Install dependencies above. Then just clone the repo inside and use ./zcutil/build.sh to build—and everything will work. So yes, the required dependencies are indeed minimal.

@TheComputerGenie
Copy link
Copy Markdown

yes, I meant toolchain not toolchest (not enough coffee yet, I guess).

DeckerSU and others added 3 commits June 12, 2025 02:53
@ca333 ca333 marked this pull request as ready for review July 17, 2025 12:55
@ca333 ca333 requested review from cipig, gcharang and smk762 July 17, 2025 12:58
Copy link
Copy Markdown

@smk762 smk762 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm. builds and launches fine locally. and only minor updates since last deep test.
MacOS CI/CD is failing, but this is not a blocker. I've drafted a PR with initial attempt to resolve, will follow up with it tommorow. along with sync tests

@ca333 ca333 merged commit 934d0d9 into master Jul 18, 2025
21 of 23 checks passed
TheComputerGenie pushed a commit to ComputerGenieCo/komodo that referenced this pull request Sep 12, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants