Issue #16003: add kafka to no-error testing by romani · Pull Request #16032 · checkstyle/checkstyle

romani · 2024-12-13T19:22:37Z

to disable all CIs:

git rm .circleci/config.yml
git rm .cirrus.yml
git rm .github/dependabot.yml
git rm .github/workflows/*
git rm .semaphore/semaphore.yml
git rm .travis.yml
git rm appveyor.yml
git rm azure-pipelines.yml
git rm codeship-services.yml
git rm codeship-steps.yml
git rm crowdin.yml

romani · 2024-12-13T20:38:15Z

https://app.circleci.com/pipelines/github/checkstyle/checkstyle/29120/workflows/f3f1942a-9861-4388-b074-a4572f6cc5ce/jobs/740365

> Task :clients:checkstyleMain

The message received from the daemon indicates that the daemon has disappeared.
Build request sent: Build{id=6b87f825-203d-434a-a42d-2afb399edc39, currentDir=/home/circleci/project/.ci-temp/kafka}
Attempting to read last messages from the daemon log...
Daemon pid: 364
  log file: /home/circleci/.gradle/daemon/8.10.2/daemon-364.out.log
----- Last 20 lines from daemon log file - daemon-364.out.log -----
2024-12-13T19:24:27.918+0000 [DEBUG] [org.gradle.launcher.daemon.server.DaemonStateCoordinator] resetting idle timer
2024-12-13T19:24:27.919+0000 [DEBUG] [org.gradle.launcher.daemon.server.DaemonStateCoordinator] daemon is running. Sleeping until state changes.
2024-12-13T19:24:27.919+0000 [INFO] [org.gradle.launcher.daemon.server.exec.StartBuildOrRespondWithBusy] Daemon is about to start building Build{id=6b87f825-203d-434a-a42d-2afb399edc39, currentDir=/home/circleci/project/.ci-temp/kafka}. Dispatching build started information...
2024-12-13T19:24:27.920+0000 [DEBUG] [org.gradle.launcher.daemon.server.SynchronizedDispatchConnection] thread 24: dispatching org.gradle.launcher.daemon.protocol.BuildStarted@2ad55e1b
2024-12-13T19:24:27.921+0000 [DEBUG] [org.gradle.launcher.daemon.server.exec.EstablishBuildEnvironment] Configuring env variables: [PATH, container, MILL_VERSION, NO_PROXY, CIRCLE_WORKFLOW_WORKSPACE_ID, SBT_VERSION, CIRCLE_PR_USERNAME, CIRCLE_PULL_REQUEST, CIRCLE_PROJECT_REPONAME, CIRCLE_OIDC_TOKEN_V2, CIRCLE_WORKING_DIRECTORY, CIRCLE_INTERNAL_TASK_DATA, PWD, LANGUAGE, GRADLE_VERSION, BASH_ENV, CIRCLE_BUILD_NUM, PAGER, CI_PULL_REQUEST, MAVEN_VERSION, COMPOSE_VER, CIRCLE_SHA1, OLDPWD, CIRCLE_NODE_INDEX, CIRCLE_NODE_TOTAL, DEBIAN_FRONTEND, LC_ALL, CIRCLE_SHELL_ENV, SHLVL, CIRCLE_PIPELINE_ID, COMPOSE_SWITCH_VERSION, DOCKER_VERSION, CIRCLE_PR_REPONAME, JAVA_HOME, TERM, CIRCLE_PROJECT_USERNAME, LANG, CIRCLE_BUILD_URL, CIRCLE_INTERNAL_SCRATCH, CIRCLE_BRANCH, CIRCLE_PULL_REQUESTS, CIRCLE_ORGANIZATION_ID, JAVA_VERSION, _, CIRCLECI, CIRCLE_PR_NUMBER, CIRCLE_USERNAME, CIRCLE_OIDC_TOKEN, CIRCLE_REPOSITORY_URL, CI, CIRCLE_JOB, CIRCLE_WORKFLOW_ID, SSH_AUTH_SOCK, CIRCLE_PROJECT_ID, CIRCLE_WORKFLOW_JOB_ID, HOME]
2024-12-13T19:24:27.925+0000 [DEBUG] [org.gradle.launcher.daemon.server.exec.LogToClient] About to start relaying all logs to the client via the connection.
2024-12-13T19:24:27.925+0000 [INFO] [org.gradle.launcher.daemon.server.exec.LogToClient] The client will now receive all logging from the daemon (pid: 364). The daemon log file: /home/circleci/.gradle/daemon/8.10.2/daemon-364.out.log
2024-12-13T19:24:27.927+0000 [INFO] [org.gradle.launcher.daemon.server.exec.LogAndCheckHealth] Starting build in new daemon [memory: 3.5 GiB]
2024-12-13T19:24:27.929+0000 [DEBUG] [org.gradle.launcher.daemon.server.exec.ExecuteBuild] The daemon has started executing the build.
2024-12-13T19:24:27.930+0000 [DEBUG] [org.gradle.launcher.daemon.server.exec.ExecuteBuild] Executing build with daemon context: DefaultDaemonContext[uid=e7a23401-a800-4a5d-97f1-30fa519ff301,javaHome=/usr/local/jdk-21.0.5,javaVersion=21,javaVendor=Eclipse Adoptium,daemonRegistryDir=/home/circleci/.gradle/daemon,pid=364,idleTimeout=10800000,priority=NORMAL,applyInstrumentationAgent=true,nativeServicesMode=ENABLED,daemonOpts=-Xss4m,-XX:+UseParallelGC,--add-opens=java.base/java.util=ALL-UNNAMED,--add-opens=java.base/java.lang=ALL-UNNAMED,--add-opens=java.base/java.lang.invoke=ALL-UNNAMED,--add-opens=java.prefs/java.util.prefs=ALL-UNNAMED,--add-exports=jdk.compiler/com.sun.tools.javac.api=ALL-UNNAMED,--add-exports=jdk.compiler/com.sun.tools.javac.util=ALL-UNNAMED,--add-opens=java.base/java.nio.charset=ALL-UNNAMED,--add-opens=java.base/java.net=ALL-UNNAMED,--add-opens=java.base/java.util.concurrent.atomic=ALL-UNNAMED,-Xmx4g,-Dfile.encoding=UTF-8,-Duser.country=US,-Duser.language=en,-Duser.variant]
Starting build with version 4.0.0-SNAPSHOT (commit id 497f5004) using Gradle 8.10.2, Java 21 and Scala 2.13.15
Build properties: ignoreFailures=false, maxParallelForks=2, maxScalacThreads=2, maxTestRetries=0
MessageGenerator: processed 25 Kafka message JSON files(s).
MessageGenerator: processed 186 Kafka message JSON files(s).
MessageGenerator: processed 1 Kafka message JSON files(s).
MessageGenerator: processed 5 Kafka message JSON files(s).
MessageGenerator: processed 2 Kafka message JSON files(s).
MessageGenerator: processed 32 Kafka message JSON files(s).
MessageGenerator: processed 4 Kafka message JSON files(s).
MessageGenerator: processed 1 Kafka message JSON files(s).----- End of the daemon log -----


FAILURE: Build failed with an exception.

* What went wrong:
Gradle build daemon disappeared unexpectedly (it may have been killed or may have crashed)

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
> Get more help at https://help.gradle.org.

so tried to configure no-daemon

romani · 2024-12-13T21:24:30Z

@dejan2609 , do you know what might be wrong with execution in CI (CircleCI) ?

gradle daemon is dying https://app.circleci.com/pipelines/github/checkstyle/checkstyle/29121/workflows/5601f52a-ff56-4c64-b85e-8c6195aa89c7/jobs/740434

on local it worked.

dejan2609 · 2024-12-14T11:59:52Z

@romani Gradle daemon strikes again 🥲 Don't worry, I think I got you covered 🛡️

You could try few things here:

Do you actually need a Gradle daemon to run ? In some cases daemon makes no sense and should be deactivated (that is: in scenarios where build container is spawned for each build). On top of that: although daemon is enabled by a default for a quite some time some people still report recent issues:
- Do not recommend to disable the daemon on CI gradle/gradle#2824 (comment)
- https://stackoverflow.com/questions/37171043/gradle-build-daemon-disappeared-unexpectedly-it-may-have-been-killed-or-may-hav/77528123#77528123
add --stacktrace --info --console=plain in order to provide more verbose output (as per: Gradle build daemon disappeared unexpectedly (it may have been killed or may have crashed) for ubuntu:20.04 gradle/gradle#17921 (comment))
increase the heap size:

⚠️ Pitfalls / things to keep in mind:

jvmargs vs. GRADLE_OPTS: Adding JVM args through GRADLE_OPTS overwrites gradle.properties gradle/gradle#19501 (comment)
Kafka is a Scala/Java mixed project (and hence even more configurations could be required): https://stackoverflow.com/questions/47454257/how-to-increase-maximum-heap-size-for-the-gradle-daemon/73660673#73660673

romani · 2024-12-16T16:52:12Z

Interesting line https://github.com/apache/kafka/blob/4aee33d6ab1345243e426e05388f6fc512970e93/.github/workflows/build.yml#L105

romani · 2024-12-21T15:15:10Z

Started Gradle worker daemon (1.094 secs) with fork options DaemonForkOptions{executable=/usr/local/jdk-21.0.5/bin/java, minHeapSize=null, maxHeapSize=2048m, jvmArgs=[-Xss4m, -XX:+UseParallelGC, --add-opens=java.base/java.io=ALL-UNNAMED, --add-opens=java.base/java.lang=ALL-UNNAMED, --add-opens=java.base/java.nio=ALL-UNNAMED, --add-opens=java.base/java.nio.file=ALL-UNNAMED, --add-opens=java.base/java.util=ALL-UNNAMED, --add-opens=java.base/java.util.concurrent=ALL-UNNAMED, --add-opens=java.base/java.util.regex=ALL-UNNAMED, --add-opens=java.base/java.util.stream=ALL-UNNAMED, --add-opens=java.base/java.text=ALL-UNNAMED, --add-opens=java.base/java.time=ALL-UNNAMED, --add-opens=java.security.jgss/sun.security.krb5=ALL-UNNAMED], keepAliveMode=DAEMON}.
This JVM does not support getting OS memory, so no OS memory status updates will be broadcast
Non-compiled module 'compiler-bridge' for Scala 2.13.15. Compiling...
  Compilation completed in 20.497s.
Compiling with Zinc Scala compiler.
Prepared Zinc Scala inputs: 0.071 secs
compiling 158 Scala sources and 20 Java sources to /home/circleci/project/.ci-temp/kafka/core/build/classes/scala/main ...----- End of the daemon log -----


FAILURE: Build failed with an exception.

* What went wrong:
Gradle build daemon disappeared unexpectedly (it may have been killed or may have crashed)

Started Gradle worker daemon (1.094 secs) with fork options DaemonForkOptions{executable=/usr/local/jdk-21.0.5/bin/java, minHeapSize=null, maxHeapSize=2048m

daemon heap size might be small.

romani · 2024-12-21T16:09:00Z

@dejan2609 , I tried bunch, nothing helps to extend memory for daemon.
Do you see how to make it?

you can copy commits in this PR and make your PR, to frequently experiment. All CIs are disabled except for kafka.

dejan2609 · 2024-12-23T14:42:37Z

@romani got this, will try to squeeze something.

romani · 2025-01-18T15:41:15Z

@dejan2609 , I hope to get some help from you on how to run build successfully

dejan2609 · 2025-01-19T12:27:38Z

No problem @romani I have this on my mind.

romani · 2025-02-02T14:54:22Z

Github, generate report for Header/Example2

github-actions · 2025-02-02T16:05:36Z

Report for Header/Example2:
https://checkstyle-diff-reports.s3.us-east-2.amazonaws.com/edc963d378736d88a68349cfc1e2c5f51d3b5e22_2025160520/reports/diff/index.html

romani · 2025-02-04T02:25:54Z

@dejan2609 , any hints from you will help us a lot.

dejan2609 · 2025-02-05T20:24:02Z

@romani solving private stuff, will start to allocate some time for this next week.

romani · 2025-02-19T17:17:03Z

@dejan2609 , please give us some hints

dejan2609 · 2025-02-22T16:39:03Z

@romani I had a lot of "you plan, God laughs on it" situations recently and hence this delay.

Can't promise fast solution but at least I started my own branch (with your commits).
Will update you there when I have something to show.

romani · 2025-02-22T17:32:53Z

@dejan2609, no worries, we operate on "eventually done" , usually by crowd sourcing and some time we make it done.
We can use any CI for this job, whatever is easiest.

romani · 2025-11-30T15:19:43Z

@stoyanK7 , if you have time to help us here , it will be huge impact to project.
Partnership with Kafka project is stuck in this PR. This is not easy update, but collaboration of 3 of us can make it.

@dejan2609 , are still around to help us in case there is Kafka specific questions ?

stoyanK7 · 2025-12-01T08:54:48Z

@romani I'll look into it 🚀

dejan2609 · 2025-12-03T08:42:33Z

Sure, I will help.

romani · 2025-12-26T03:59:05Z

done in other PR

romani force-pushed the i16003-kafka branch from 526bd70 to 6d4ca88 Compare December 13, 2024 20:36

romani force-pushed the i16003-kafka branch from 6d4ca88 to f2d1c12 Compare December 21, 2024 15:05

romani force-pushed the i16003-kafka branch 2 times, most recently from 13400a7 to 07de29e Compare December 21, 2024 15:26

romani added 3 commits December 21, 2024 07:34

minor: DO NOT MERGE, disable all CIs

591e0f2

minor: DO NOT MERGE, disable all circleci except for kafka

9c63a1c

Issue checkstyle#16003: add kafka to no-error testing

ae773b2

romani force-pushed the i16003-kafka branch from 07de29e to ae773b2 Compare December 21, 2024 15:38

romani added 2 commits December 21, 2024 07:48

minor: try

7f2709d

minor: try2

edc963d

romani mentioned this pull request Dec 28, 2024

Issue #8244: Added expandable menu for mobile/tablet devices #16066

Merged

romani mentioned this pull request Feb 2, 2025

Workflow regression-report.yml should support of config bundles with extra configuration files (Header check) #16121

Closed

dejan2609 mentioned this pull request Feb 22, 2025

[WIP based on PR #16032] - Issue #16003: add kafka to no-error testing #16395

Closed

stoyanK7 mentioned this pull request Dec 9, 2025

Issue #16003: Add Kafka to no-error testing #18263

Merged

romani closed this Dec 26, 2025

Uh oh!

Conversation

romani commented Dec 13, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

romani commented Dec 13, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

romani commented Dec 13, 2024

Uh oh!

dejan2609 commented Dec 14, 2024

Uh oh!

romani commented Dec 16, 2024

Uh oh!

romani commented Dec 21, 2024

Uh oh!

romani commented Dec 21, 2024

Uh oh!

dejan2609 commented Dec 23, 2024

Uh oh!

romani commented Jan 18, 2025

Uh oh!

dejan2609 commented Jan 19, 2025

Uh oh!

romani commented Feb 2, 2025

Uh oh!

github-actions Bot commented Feb 2, 2025

Uh oh!

romani commented Feb 4, 2025

Uh oh!

dejan2609 commented Feb 5, 2025

Uh oh!

romani commented Feb 19, 2025

Uh oh!

dejan2609 commented Feb 22, 2025

Uh oh!

romani commented Feb 22, 2025

Uh oh!

romani commented Nov 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

stoyanK7 commented Dec 1, 2025

Uh oh!

dejan2609 commented Dec 3, 2025

Uh oh!

romani commented Dec 26, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

romani commented Dec 13, 2024 •

edited

Loading

romani commented Dec 13, 2024 •

edited

Loading

romani commented Nov 30, 2025 •

edited

Loading