Conversation
7b03f25 to
3846224
Compare
cad8cc5 to
7dd9560
Compare
JohnMalmberg
left a comment
There was a problem hiding this comment.
This code does not appear to be able to pass groovylint checks.
Every "def" declaration is going to cause Jenkins log noise. There are a few cases where it must be used for some function return values, and in those cases it should be suppressed, in all others it should be the proper type.
The groovylint for visual studio code can not detect if the first use of a variable is missing a type declaration, which is serious for Jenkins as can cause intermittent data corruption.
The current system-pipeline-lib PR I have open has code to doing the groovylint, that also checks for undeclared variables both runnable as a github action and as a pre-commit hook.
|
@JohnMalmberg wrote:
That's great! When it got landed in the system-pipeline-lib we can adopt it here and forget about this kind of issues in the future. 🙂 Ref: https://github.com/daos-stack/system-pipeline-lib/pull/18 |
Jenkinsfile
Outdated
| def proxy = env.DAOS_HTTPS_PROXY | ||
| def idx = proxy.lastIndexOf(':') | ||
| def host = proxy.substring(0, idx) | ||
| def port = proxy.substring(idx + 1) | ||
| def vars = [ | ||
| '${HTTP_HOST}': host, | ||
| '${HTTP_PORT}': port, | ||
| '${HTTPS_HOST}': host, | ||
| '${HTTPS_PORT}': port, | ||
| ] | ||
|
|
||
| def properties = readFile('gradle.properties.template') |
There was a problem hiding this comment.
| def proxy = env.DAOS_HTTPS_PROXY | |
| def idx = proxy.lastIndexOf(':') | |
| def host = proxy.substring(0, idx) | |
| def port = proxy.substring(idx + 1) | |
| def vars = [ | |
| '${HTTP_HOST}': host, | |
| '${HTTP_PORT}': port, | |
| '${HTTPS_HOST}': host, | |
| '${HTTPS_PORT}': port, | |
| ] | |
| def properties = readFile('gradle.properties.template') | |
| String proxy = env.DAOS_HTTPS_PROXY | |
| String idx = proxy.lastIndexOf(':') | |
| String host = proxy.substring(0, idx) | |
| String port = proxy.substring(idx + 1) | |
| Map vars = [ | |
| '${HTTP_HOST}': host, | |
| '${HTTP_PORT}': port, | |
| '${HTTPS_HOST}': host, | |
| '${HTTPS_PORT}': port, | |
| ] | |
| String properties = readFile('gradle.properties.template') |
There was a problem hiding this comment.
@JohnMalmberg is it enough to fix the def issue?
There was a problem hiding this comment.
Looks like that may be. Also need to avoid going through the corporate proxy, even if we have to add mirrors to our artifact servers.
JohnMalmberg
left a comment
There was a problem hiding this comment.
Looks like this still needs some restructuring.
We should be maintaining the jar file in the artifact server in a local repository if we generate it, or should have the artifact server be mirroring the repository that the jar is in.
All use of the corporate proxy should be avoided. A corporate proxy is not designed to handle the network needs of a CI system, and should be avoided as much as possible.
There was a problem hiding this comment.
A jar file is an artifact, why are we storing a jar file in a git repository?
CI should be fetching artifacts from the locally hosted artifact server, do we need a proxy repository setup for gradle/java related repositories?
There was a problem hiding this comment.
@JohnMalmberg wrote:
A jar file is an artifact, why are we storing a jar file in a git repository?
I am not a big fan either. But this is how Gradle suggest to deal with it:
This task also generates a small gradle-wrapper.jar bootstrap JAR file and properties file which should also be committed to your VCS.
Ref: https://docs.gradle.org/current/dsl/org.gradle.api.tasks.wrapper.Wrapper.html
Alternatively we could fall back to use gradle directly. gradlew is just a little bit more convenient since it makes sure all of us (CI included) use the exact same version of gradle and this version is already committed into the repository (please see gradle/wrapper/gradle-wrapper.properties).
If it makes any difference, this gradle/wrapper/gradle-wrapper.jar file is also designed to be small and do not change very often. So, if agree to commit this now we ought to be set up for life. 😁
There was a problem hiding this comment.
This still may be the wrong place for it for total cost of operation, even though that jar file according to internet searches is only 45 kb in size. And data that can change being in a support library is a ticking timebomb for future maintenance.
In spite of Gradle documentation suggesting it as a best practice, For a CI environment, it is better to create a local repository in Artifactory to pull a prebuilt jar and other files into the work space when it is needed than to have it present in every workspace for every job on in Jenkins.
If we are pulling this gradle-wrapper.jar file from somewhere, we can probably mirror that location in Artifactory, and if needed provide an environment variable for jobs that need it to find it.
If we are building it locally, then the build and any self test should be in a Jenkins job of some type, and for now manually copied to a local repository in the Artifactory server.
.gitignore
Outdated
| @@ -0,0 +1,4 @@ | |||
| .gradle | |||
| build | |||
| *.jar | |||
There was a problem hiding this comment.
We have a .jar file in this review, yet jar files are excluded from showing up? Seems contradictory.
There was a problem hiding this comment.
I thought it wouldn't be necessary to put this file in the repository.
There was a problem hiding this comment.
All source files should be pulled from a local artifact server instead going through a proxy. If we do not have this in our artifact server, we need to add it to the artifact server.
The corporate proxy needs to be avoided as much as possible to reduce loading on it and improve CI robustness.
Jenkinsfile
Outdated
| def proxy = env.DAOS_HTTPS_PROXY | ||
| def idx = proxy.lastIndexOf(':') | ||
| def host = proxy.substring(0, idx) | ||
| def port = proxy.substring(idx + 1) | ||
| def vars = [ | ||
| '${HTTP_HOST}': host, | ||
| '${HTTP_PORT}': port, | ||
| '${HTTPS_HOST}': host, | ||
| '${HTTPS_PORT}': port, | ||
| ] | ||
|
|
||
| def properties = readFile('gradle.properties.template') |
There was a problem hiding this comment.
Looks like that may be. Also need to avoid going through the corporate proxy, even if we have to add mirrors to our artifact servers.
There was a problem hiding this comment.
This still may be the wrong place for it for total cost of operation, even though that jar file according to internet searches is only 45 kb in size. And data that can change being in a support library is a ticking timebomb for future maintenance.
In spite of Gradle documentation suggesting it as a best practice, For a CI environment, it is better to create a local repository in Artifactory to pull a prebuilt jar and other files into the work space when it is needed than to have it present in every workspace for every job on in Jenkins.
If we are pulling this gradle-wrapper.jar file from somewhere, we can probably mirror that location in Artifactory, and if needed provide an environment variable for jobs that need it to find it.
If we are building it locally, then the build and any self test should be in a Jenkins job of some type, and for now manually copied to a local repository in the Artifactory server.
Signed-off-by: Oksana Salyk <oksana.salyk@hpe.com> update after review Signed-off-by: Oksana Salyk <oksana.salyk@hpe.com> test: add another batch of fixes Signed-off-by: Oksana Salyk <oksana.salyk@hpe.com> improve the next tests Signed-off-by: Oksana Salyk <oksana.salyk@hpe.com>
Signed-off-by: Oksana Salyk <oksana.salyk@hpe.com> add gradlew configuration Signed-off-by: Oksana Salyk <oksana.salyk@hpe.com> add corrections after review Signed-off-by: Oksana Salyk <oksana.salyk@hpe.com> fix stage name Signed-off-by: Oksana Salyk <oksana.salyk@hpe.com> fix stage name Signed-off-by: Oksana Salyk <oksana.salyk@hpe.com> Skip-daos-build-and-test: true
Skip-daos-build-and-test: true Signed-off-by: Oksana Salyk <oksana.salyk@hpe.com>
Signed-off-by: Oksana Salyk <oksana.salyk@hpe.com>
Skip-daos-build-and-test: true Signed-off-by: Oksana Salyk <oksana.salyk@hpe.com>
Skip-daos-build-and-test: true Signed-off-by: Oksana Salyk <oksana.salyk@hpe.com>
Skip-daos-build-and-test: true Signed-off-by: Oksana Salyk <oksana.salyk@hpe.com>
Skip-daos-build-and-test: true Signed-off-by: Oksana Salyk <oksana.salyk@hpe.com>
Signed-off-by: Oksana Salyk <oksana.salyk@hpe.com>
Skip-daos-build-and-test: true Signed-off-by: Oksana Salyk <oksana.salyk@hpe.com>
Skip-daos-build-and-test: true Signed-off-by: Oksana Salyk <oksana.salyk@hpe.com>
Signed-off-by: Oksana Salyk <oksana.salyk@hpe.com>
Signed-off-by: Oksana Salyk <oksana.salyk@hpe.com>
Signed-off-by: Oksana Salyk <oksana.salyk@hpe.com>
c4f574f to
40b0471
Compare
Signed-off-by: Tomasz Gromadzki <tomasz.gromadzki@hpe.com>
| /* groovylint-disable DuplicateListLiteral, DuplicateMapLiteral, DuplicateNumberLiteral */ | ||
| // groovylint-disable DuplicateStringLiteral, NestedBlockDepth, VariableName | ||
| /* Copyright 2019-2024 Intel Corporation | ||
| * Copyright 2025 Hewlett Packard Enterprise Development LP |
There was a problem hiding this comment.
Please update.
| * Copyright 2025-2026 Hewlett Packard Enterprise Development LP |
|
It seems none of us is actually comfortable with diff --git a/.gitignore b/.gitignore
index fc2f44d..79c16f7 100644
--- a/.gitignore
+++ b/.gitignore
@@ -1,3 +1,4 @@
.gradle
build
+gradle
gradle.properties
diff --git a/Jenkinsfile b/Jenkinsfile
index 2f12e8b..9d9056f 100755
--- a/Jenkinsfile
+++ b/Jenkinsfile
@@ -151,23 +151,10 @@ pipeline {
label 'JUnit_jdk_tests'
}
steps {
- script {
- String proxy = env.DAOS_HTTPS_PROXY
- int idx = proxy.lastIndexOf(':')
- String host = proxy.substring(0, idx)
- String port = proxy.substring(idx + 1)
- Map vars = [
- '${HTTP_HOST}': host,
- '${HTTP_PORT}': port,
- '${HTTPS_HOST}': host,
- '${HTTPS_PORT}': port,
- ]
-
- String properties = readFile('gradle.properties.template')
- vars.each { k, v -> properties = properties.replace(k, v) }
- writeFile file: 'gradle.properties', text: properties
- }
- sh './gradlew spotlessCheck test --no-daemon'
+ sh '''
+ .gradle-init.sh
+ ./gradle spotlessCheck test --no-daemon
+ '''
}
post {
always {
diff --git a/gradle-init.sh b/gradle-init.sh
new file mode 100755
index 0000000..a641f5d
--- /dev/null
+++ b/gradle-init.sh
@@ -0,0 +1,39 @@
+#!/usr/bin/env bash
+#
+# Copyright 2026, Hewlett Packard Enterprise Development LP
+#
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+#
+set -euo pipefail
+
+DIST_FILE='gradle-9.0-bin.zip'
+DIST_URL="https://artifactory.daos.hpc.amslabs.hpecorp.net/artifactory/gradle-services-proxy/distributions/$DIST_FILE"
+DIST_DIR='gradle-9.0.0' # it must match the name of the directory stored inside the ZIP
+
+GRADLE_USER_HOME='.gradle'
+
+mkdir -p "$GRADLE_USER_HOME"
+
+DIR="$GRADLE_USER_HOME/$DIST_DIR"
+BIN="$DIR/bin/gradle"
+if [ -f "$BIN" ]; then
+ echo "WARN: '$BIN' is already there."
+ echo
+ echo "Remove '$DIR' if you want to download it again."
+else
+ # download and unzip
+ cd "$GRADLE_USER_HOME"
+ wget "$DIST_URL"
+ unzip "$DIST_FILE"
+ rm "$DIST_FILE"
+ cd -
+
+ if [ ! -f "$BIN" ]; then
+ echo "ERROR: There is no '$BIN' file in '$DIST_URL'."
+ exit 1
+ fi
+fi
+
+SYM=gradle
+rm -f $SYM
+ln -s "$BIN" $SYM
diff --git a/gradle.properties.template b/gradle.properties.template
deleted file mode 100644
index 1ebcb1c..0000000
--- a/gradle.properties.template
+++ /dev/null
@@ -1,4 +0,0 @@
-systemProp.http.proxyHost=${HTTP_HOST}
-systemProp.http.proxyPort=${HTTP_PORT}
-systemProp.https.proxyHost=${HTTPS_HOST}
-systemProp.https.proxyPort=${HTTPS_PORT}
diff --git a/gradle/wrapper/gradle-wrapper.jar b/gradle/wrapper/gradle-wrapper.jar
deleted file mode 100644
index e644113..0000000
Binary files a/gradle/wrapper/gradle-wrapper.jar and /dev/null differ
diff --git a/gradle/wrapper/gradle-wrapper.properties b/gradle/wrapper/gradle-wrapper.properties
deleted file mode 100644
index f4bb8d5..0000000
--- a/gradle/wrapper/gradle-wrapper.properties
+++ /dev/null
@@ -1,7 +0,0 @@
-distributionBase=GRADLE_USER_HOME
-distributionPath=wrapper/dists
-distributionUrl=https\://artifactory.daos.hpc.amslabs.hpecorp.net/artifactory/gradle-services-proxy/distributions/gradle-9.0-bin.zip
-networkTimeout=10000
-validateDistributionUrl=true
-zipStoreBase=GRADLE_USER_HOME
-zipStorePath=wrapper/distsBonus is we can remove |
No description provided.