Skip to content

add unit test to unitTest.groovy#500

Open
osalyk wants to merge 15 commits intomasterfrom
osalyk/pipline-lib_tests
Open

add unit test to unitTest.groovy#500
osalyk wants to merge 15 commits intomasterfrom
osalyk/pipline-lib_tests

Conversation

@osalyk
Copy link
Copy Markdown
Contributor

@osalyk osalyk commented Mar 17, 2026

No description provided.

@osalyk osalyk force-pushed the osalyk/pipline-lib_tests branch 2 times, most recently from 7b03f25 to 3846224 Compare March 18, 2026 09:38
@osalyk osalyk marked this pull request as ready for review March 18, 2026 11:04
@osalyk osalyk force-pushed the osalyk/pipline-lib_tests branch from cad8cc5 to 7dd9560 Compare March 18, 2026 14:11
janekmi
janekmi previously approved these changes Mar 19, 2026
Copy link
Copy Markdown
Collaborator

@JohnMalmberg JohnMalmberg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This code does not appear to be able to pass groovylint checks.

Every "def" declaration is going to cause Jenkins log noise. There are a few cases where it must be used for some function return values, and in those cases it should be suppressed, in all others it should be the proper type.

The groovylint for visual studio code can not detect if the first use of a variable is missing a type declaration, which is serious for Jenkins as can cause intermittent data corruption.

The current system-pipeline-lib PR I have open has code to doing the groovylint, that also checks for undeclared variables both runnable as a github action and as a pre-commit hook.

@janekmi
Copy link
Copy Markdown
Contributor

janekmi commented Mar 19, 2026

@JohnMalmberg wrote:

The current system-pipeline-lib PR I have open has code to doing the groovylint, that also checks for undeclared variables both runnable as a github action and as a pre-commit hook.

That's great! When it got landed in the system-pipeline-lib we can adopt it here and forget about this kind of issues in the future. 🙂

Ref: https://github.com/daos-stack/system-pipeline-lib/pull/18

Jenkinsfile Outdated
Comment on lines +155 to +166
def proxy = env.DAOS_HTTPS_PROXY
def idx = proxy.lastIndexOf(':')
def host = proxy.substring(0, idx)
def port = proxy.substring(idx + 1)
def vars = [
'${HTTP_HOST}': host,
'${HTTP_PORT}': port,
'${HTTPS_HOST}': host,
'${HTTPS_PORT}': port,
]

def properties = readFile('gradle.properties.template')
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
def proxy = env.DAOS_HTTPS_PROXY
def idx = proxy.lastIndexOf(':')
def host = proxy.substring(0, idx)
def port = proxy.substring(idx + 1)
def vars = [
'${HTTP_HOST}': host,
'${HTTP_PORT}': port,
'${HTTPS_HOST}': host,
'${HTTPS_PORT}': port,
]
def properties = readFile('gradle.properties.template')
String proxy = env.DAOS_HTTPS_PROXY
String idx = proxy.lastIndexOf(':')
String host = proxy.substring(0, idx)
String port = proxy.substring(idx + 1)
Map vars = [
'${HTTP_HOST}': host,
'${HTTP_PORT}': port,
'${HTTPS_HOST}': host,
'${HTTPS_PORT}': port,
]
String properties = readFile('gradle.properties.template')

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@JohnMalmberg is it enough to fix the def issue?

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like that may be. Also need to avoid going through the corporate proxy, even if we have to add mirrors to our artifact servers.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

Copy link
Copy Markdown
Collaborator

@JohnMalmberg JohnMalmberg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like this still needs some restructuring.

We should be maintaining the jar file in the artifact server in a local repository if we generate it, or should have the artifact server be mirroring the repository that the jar is in.

All use of the corporate proxy should be avoided. A corporate proxy is not designed to handle the network needs of a CI system, and should be avoided as much as possible.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A jar file is an artifact, why are we storing a jar file in a git repository?

CI should be fetching artifacts from the locally hosted artifact server, do we need a proxy repository setup for gradle/java related repositories?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@JohnMalmberg wrote:

A jar file is an artifact, why are we storing a jar file in a git repository?

I am not a big fan either. But this is how Gradle suggest to deal with it:

This task also generates a small gradle-wrapper.jar bootstrap JAR file and properties file which should also be committed to your VCS.

Ref: https://docs.gradle.org/current/dsl/org.gradle.api.tasks.wrapper.Wrapper.html

Alternatively we could fall back to use gradle directly. gradlew is just a little bit more convenient since it makes sure all of us (CI included) use the exact same version of gradle and this version is already committed into the repository (please see gradle/wrapper/gradle-wrapper.properties).

If it makes any difference, this gradle/wrapper/gradle-wrapper.jar file is also designed to be small and do not change very often. So, if agree to commit this now we ought to be set up for life. 😁

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This still may be the wrong place for it for total cost of operation, even though that jar file according to internet searches is only 45 kb in size. And data that can change being in a support library is a ticking timebomb for future maintenance.

In spite of Gradle documentation suggesting it as a best practice, For a CI environment, it is better to create a local repository in Artifactory to pull a prebuilt jar and other files into the work space when it is needed than to have it present in every workspace for every job on in Jenkins.

If we are pulling this gradle-wrapper.jar file from somewhere, we can probably mirror that location in Artifactory, and if needed provide an environment variable for jobs that need it to find it.

If we are building it locally, then the build and any self test should be in a Jenkins job of some type, and for now manually copied to a local repository in the Artifactory server.

.gitignore Outdated
@@ -0,0 +1,4 @@
.gradle
build
*.jar
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We have a .jar file in this review, yet jar files are excluded from showing up? Seems contradictory.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I thought it wouldn't be necessary to put this file in the repository.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All source files should be pulled from a local artifact server instead going through a proxy. If we do not have this in our artifact server, we need to add it to the artifact server.

The corporate proxy needs to be avoided as much as possible to reduce loading on it and improve CI robustness.

Jenkinsfile Outdated
Comment on lines +155 to +166
def proxy = env.DAOS_HTTPS_PROXY
def idx = proxy.lastIndexOf(':')
def host = proxy.substring(0, idx)
def port = proxy.substring(idx + 1)
def vars = [
'${HTTP_HOST}': host,
'${HTTP_PORT}': port,
'${HTTPS_HOST}': host,
'${HTTPS_PORT}': port,
]

def properties = readFile('gradle.properties.template')
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like that may be. Also need to avoid going through the corporate proxy, even if we have to add mirrors to our artifact servers.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This still may be the wrong place for it for total cost of operation, even though that jar file according to internet searches is only 45 kb in size. And data that can change being in a support library is a ticking timebomb for future maintenance.

In spite of Gradle documentation suggesting it as a best practice, For a CI environment, it is better to create a local repository in Artifactory to pull a prebuilt jar and other files into the work space when it is needed than to have it present in every workspace for every job on in Jenkins.

If we are pulling this gradle-wrapper.jar file from somewhere, we can probably mirror that location in Artifactory, and if needed provide an environment variable for jobs that need it to find it.

If we are building it locally, then the build and any self test should be in a Jenkins job of some type, and for now manually copied to a local repository in the Artifactory server.

osalyk added 10 commits March 31, 2026 11:13
Signed-off-by: Oksana Salyk <oksana.salyk@hpe.com>

update after review

Signed-off-by: Oksana Salyk <oksana.salyk@hpe.com>

test: add another batch of fixes

Signed-off-by: Oksana Salyk <oksana.salyk@hpe.com>

improve the next tests

Signed-off-by: Oksana Salyk <oksana.salyk@hpe.com>
Signed-off-by: Oksana Salyk <oksana.salyk@hpe.com>

add gradlew configuration

Signed-off-by: Oksana Salyk <oksana.salyk@hpe.com>

add corrections after review

Signed-off-by: Oksana Salyk <oksana.salyk@hpe.com>

fix stage name

Signed-off-by: Oksana Salyk <oksana.salyk@hpe.com>

fix stage name

Signed-off-by: Oksana Salyk <oksana.salyk@hpe.com>

Skip-daos-build-and-test: true
Skip-daos-build-and-test: true

Signed-off-by: Oksana Salyk <oksana.salyk@hpe.com>
Signed-off-by: Oksana Salyk <oksana.salyk@hpe.com>
Skip-daos-build-and-test: true

Signed-off-by: Oksana Salyk <oksana.salyk@hpe.com>
Skip-daos-build-and-test: true

Signed-off-by: Oksana Salyk <oksana.salyk@hpe.com>
Skip-daos-build-and-test: true

Signed-off-by: Oksana Salyk <oksana.salyk@hpe.com>
Skip-daos-build-and-test: true

Signed-off-by: Oksana Salyk <oksana.salyk@hpe.com>
Signed-off-by: Oksana Salyk <oksana.salyk@hpe.com>
Skip-daos-build-and-test: true

Signed-off-by: Oksana Salyk <oksana.salyk@hpe.com>
osalyk added 4 commits March 31, 2026 11:13
Skip-daos-build-and-test: true

Signed-off-by: Oksana Salyk <oksana.salyk@hpe.com>
Signed-off-by: Oksana Salyk <oksana.salyk@hpe.com>
Signed-off-by: Oksana Salyk <oksana.salyk@hpe.com>
Signed-off-by: Oksana Salyk <oksana.salyk@hpe.com>
@osalyk osalyk force-pushed the osalyk/pipline-lib_tests branch from c4f574f to 40b0471 Compare March 31, 2026 09:14
Signed-off-by: Tomasz Gromadzki <tomasz.gromadzki@hpe.com>
/* groovylint-disable DuplicateListLiteral, DuplicateMapLiteral, DuplicateNumberLiteral */
// groovylint-disable DuplicateStringLiteral, NestedBlockDepth, VariableName
/* Copyright 2019-2024 Intel Corporation
* Copyright 2025 Hewlett Packard Enterprise Development LP
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please update.

Suggested change
* Copyright 2025-2026 Hewlett Packard Enterprise Development LP

@janekmi
Copy link
Copy Markdown
Contributor

janekmi commented Apr 3, 2026

It seems none of us is actually comfortable with gradlew and its JAR binary. So I think the best idea is to just toss it out and do it ourselves. I wrote a little script which just downloads the binary from our artifactory and creates a symlink so after calling gradle-init.sh you can call just ./gradle directly from the main directory. A little unorthodox but clean and simple.

diff --git a/.gitignore b/.gitignore
index fc2f44d..79c16f7 100644
--- a/.gitignore
+++ b/.gitignore
@@ -1,3 +1,4 @@
 .gradle
 build
+gradle
 gradle.properties
diff --git a/Jenkinsfile b/Jenkinsfile
index 2f12e8b..9d9056f 100755
--- a/Jenkinsfile
+++ b/Jenkinsfile
@@ -151,23 +151,10 @@ pipeline {
                         label 'JUnit_jdk_tests'
                     }
                     steps {
-                        script {
-                            String proxy = env.DAOS_HTTPS_PROXY
-                            int idx = proxy.lastIndexOf(':')
-                            String host = proxy.substring(0, idx)
-                            String port = proxy.substring(idx + 1)
-                            Map vars = [
-                                '${HTTP_HOST}': host,
-                                '${HTTP_PORT}': port,
-                                '${HTTPS_HOST}': host,
-                                '${HTTPS_PORT}': port,
-                            ]
-
-                            String properties = readFile('gradle.properties.template')
-                            vars.each { k, v -> properties = properties.replace(k, v) }
-                            writeFile file: 'gradle.properties', text: properties
-                        }
-                        sh './gradlew spotlessCheck test --no-daemon'
+                        sh '''
+                        .gradle-init.sh
+                        ./gradle spotlessCheck test --no-daemon
+                        '''
                     }
                     post {
                         always {
diff --git a/gradle-init.sh b/gradle-init.sh
new file mode 100755
index 0000000..a641f5d
--- /dev/null
+++ b/gradle-init.sh
@@ -0,0 +1,39 @@
+#!/usr/bin/env bash
+#
+# Copyright 2026, Hewlett Packard Enterprise Development LP
+#
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+#
+set -euo pipefail
+
+DIST_FILE='gradle-9.0-bin.zip'
+DIST_URL="https://artifactory.daos.hpc.amslabs.hpecorp.net/artifactory/gradle-services-proxy/distributions/$DIST_FILE"
+DIST_DIR='gradle-9.0.0' # it must match the name of the directory stored inside the ZIP
+
+GRADLE_USER_HOME='.gradle'
+
+mkdir -p "$GRADLE_USER_HOME"
+
+DIR="$GRADLE_USER_HOME/$DIST_DIR"
+BIN="$DIR/bin/gradle"
+if [ -f "$BIN" ]; then
+        echo "WARN: '$BIN' is already there."
+        echo
+        echo "Remove '$DIR' if you want to download it again."
+else
+        # download and unzip
+        cd "$GRADLE_USER_HOME"
+        wget "$DIST_URL"
+        unzip "$DIST_FILE"
+        rm "$DIST_FILE"
+        cd -
+
+        if [ ! -f "$BIN" ]; then
+                echo "ERROR: There is no '$BIN' file in '$DIST_URL'."
+                exit 1
+        fi
+fi
+
+SYM=gradle
+rm -f $SYM
+ln -s "$BIN" $SYM
diff --git a/gradle.properties.template b/gradle.properties.template
deleted file mode 100644
index 1ebcb1c..0000000
--- a/gradle.properties.template
+++ /dev/null
@@ -1,4 +0,0 @@
-systemProp.http.proxyHost=${HTTP_HOST}
-systemProp.http.proxyPort=${HTTP_PORT}
-systemProp.https.proxyHost=${HTTPS_HOST}
-systemProp.https.proxyPort=${HTTPS_PORT}
diff --git a/gradle/wrapper/gradle-wrapper.jar b/gradle/wrapper/gradle-wrapper.jar
deleted file mode 100644
index e644113..0000000
Binary files a/gradle/wrapper/gradle-wrapper.jar and /dev/null differ
diff --git a/gradle/wrapper/gradle-wrapper.properties b/gradle/wrapper/gradle-wrapper.properties
deleted file mode 100644
index f4bb8d5..0000000
--- a/gradle/wrapper/gradle-wrapper.properties
+++ /dev/null
@@ -1,7 +0,0 @@
-distributionBase=GRADLE_USER_HOME
-distributionPath=wrapper/dists
-distributionUrl=https\://artifactory.daos.hpc.amslabs.hpecorp.net/artifactory/gradle-services-proxy/distributions/gradle-9.0-bin.zip
-networkTimeout=10000
-validateDistributionUrl=true
-zipStoreBase=GRADLE_USER_HOME
-zipStorePath=wrapper/dists

Bonus is we can remove gradlew so no odd copyrights. 😁

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants