Skip to content

[BUG] Out of memory exeption #5378

@timofeevmd

Description

@timofeevmd

OS and Environment

Linux, AWS, k8s

GIT commit hash

f348b9a

Minimum working example / Steps to reproduce

perf report include full details list of precondition

### 0. Test objective:

  • Apply the load in the required volume.
  • Apply the load over an extended period of time.

#### 1. Infrastructure

  • iroha version: version="2.0.0-rc.1" git_commit_sha="f348b9a8"
  • java sdk version: commit: commit: efeb5a233e
  • iroha2-perf version: commit : 041736f
  • 5 peers

### SPECIAL CONDITION FOR STAND PREPARATION

  • We increased the disks for Longlive to 20 GB.

  • Ingress initially had more resources (2x horizontally) at the start of the load test to handle the peak load during the sudden start.

  • Iroha has priority; Kubernetes relocates pods within the cluster based on priorities. By default, Iroha has priority over other applications. However, services like NGINX have a higher priority, which makes sense. For that test, I increased the priority for our iroha2-test.

#### 2. images/config

### PREPARATION LONGEVITY ENV

Access to standard monitoring tools

On the perf generator

git clone https://github.com/soramitsu/iroha2-perf.git &&
git checkout iroha/2_0_0-rc_1/keypair &&
cd performance-generator/ &&
mvn -N io.takari:maven:wrapper &&
./mvnw gatling:test -Dgatling.simulationClass=simulation.transactions.rampConstant.TransferAssetSimulation -DtargetURL=  -DremoteLogin=  -DremotePassword= -DstartLevelUsers=0 -DendLevelUsers=234 -DrampDuring=4500 -DstageDuration=86400 -DmaxDuration=86401

Actual result

out of memory exeption

iroha2 logs open search

kubernetes logs

OOM
Image

Resources utilization
Image

performance metrics
Image

Expected result

The load is applied evenly throughout the entire test. There is no CPU or memory utilization.

Logs

Details Mar 20 01:23:31 ip-10-1-124-86 containerd: time="2025-03-20T01:23:31.467644696Z" level=info msg="TaskOOM event container_id:\"c59c9b3d5c12272d6c37de5d0d068ddb936b74a48e396ffab002bbeffd0a98a0\""

Who can help to reproduce?

@timofeevmd @RamilMus

Notes

No response

Metadata

Metadata

Labels

BugSomething isn't working

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions