tongbu

.github/PULL_REQUEST_TEMPLATE.md

@@ -6,7 +6,7 @@ Make sure that:
 -->
 
 - [ ] You have read the [Spring Data contribution guidelines](https://github.com/spring-projects/spring-data-build/blob/master/CONTRIBUTING.adoc).
-- [ ] There is a ticket in the bug tracker for the project in our [JIRA](https://jira.spring.io/browse/DATAES).
+- [ ] There is a ticket in the bug tracker for the project in our [issue tracker](https://github.com/spring-projects/spring-data-elasticsearch/issues).
 - [ ] You use the code formatters provided [here](https://github.com/spring-projects/spring-data-build/tree/master/etc/ide) and have them applied to your changes. Don’t submit any formatting related changes.
 - [ ] You submit test cases (unit or integration tests) that back your changes.
 - [ ] You added yourself as author in the headers of the classes you touched. Amend the date range in the Apache license header if needed. For new types, add the license header (copy from another file and set the current year only).

CI.adoc

@@ -1,15 +1,15 @@
 = Continuous Integration
 
-image:https://jenkins.spring.io/buildStatus/icon?job=spring-data-elasticsearch%2Fmaster&subject=Moore%20(master)[link=https://jenkins.spring.io/view/SpringData/job/spring-data-elasticsearch/]
-image:https://jenkins.spring.io/buildStatus/icon?job=spring-data-elasticsearch%2F3.1.x&subject=Lovelace%20(3.1.x)[link=https://jenkins.spring.io/view/SpringData/job/spring-data-elasticsearch/]
-image:https://jenkins.spring.io/buildStatus/icon?job=spring-data-elasticsearch%2F2.1.x&subject=Ingalls%20(2.1.x)[link=https://jenkins.spring.io/view/SpringData/job/spring-data-elasticsearch/]
+image:https://jenkins.spring.io/buildStatus/icon?job=spring-data-elasticsearch%2Fmaster&subject=2020.0.0%20(master)[link=https://jenkins.spring.io/view/SpringData/job/spring-data-elasticsearch/]
+image:https://jenkins.spring.io/buildStatus/icon?job=spring-data-elasticsearch%2F4.0.x&subject=Neumann%20(4.0.x)[link=https://jenkins.spring.io/view/SpringData/job/spring-data-elasticsearch/]
+image:https://jenkins.spring.io/buildStatus/icon?job=spring-data-elasticsearch%2F3.2.x&subject=Moore%20(3.2.x)[link=https://jenkins.spring.io/view/SpringData/job/spring-data-elasticsearch/]
 
 == Running CI tasks locally
 
 Since this pipeline is purely Docker-based, it's easy to:
 
 * Debug what went wrong on your local machine.
-* Test out a a tweak to your `test.sh` script before sending it out.
+* Test out a a tweak to your `verify.sh` script before sending it out.
 * Experiment against a new image before submitting your pull request.
 
 All of these use cases are great reasons to essentially run what the CI server does on your local machine.
@@ -30,7 +30,7 @@ Since the container is binding to your source, you can make edits from your IDE
 
 If you need to package things up, do this:
 
-1. `docker run -it --mount type=bind,source="$(pwd)",target=/spring-data-elasticsearch-github adoptopenjdk/openjdk8:latest /bin/bash`
+1. `docker run -it -v /var/run/docker.sock:/var/run/docker.sock --mount type=bind,source="$(pwd)",target=/spring-data-elasticsearch-github adoptopenjdk/openjdk8:latest /bin/bash`
 +
 This will launch the Docker image and mount your source code at `spring-data-elasticsearch-github`.
 +

CONTRIBUTING.adoc

@@ -1,3 +1,7 @@
 = Spring Data contribution guidelines
 
 You find the contribution guidelines for Spring Data projects https://github.com/spring-projects/spring-data-build/blob/master/CONTRIBUTING.adoc[here].
+
+== Running the test locally
+
+In order to run the tests locally with `./mvnw test` you need to have docker running because Spring Data Elasticsearch uses https://www.testcontainers.org/[Testcontainers] to start a local running Elasticsearch instance.

Jenkinsfile

@@ -20,54 +20,78 @@ pipeline {
 				}
 			}
 			agent {
-				docker {
-					image 'adoptopenjdk/openjdk8:latest'
-					label 'data'
-					args '-v $HOME:/tmp/jenkins-home'
-				}
+				label 'data'
 			}
 			options { timeout(time: 30, unit: 'MINUTES') }
+
+			environment {
+				DOCKER_HUB = credentials('hub.docker.com-springbuildmaster')
+			}
+
 			steps {
-				sh 'rm -rf ?'
-				sh 'MAVEN_OPTS="-Duser.name=jenkins -Duser.home=/tmp/jenkins-home" ./mvnw clean dependency:list test -Dsort -U -B'
+				script {
+					docker.withRegistry('', 'hub.docker.com-springbuildmaster') {
+						docker.image('adoptopenjdk/openjdk8:latest').inside('-u root -v /var/run/docker.sock:/var/run/docker.sock -v /usr/bin/docker:/usr/bin/docker -v $HOME:/tmp/jenkins-home') {
+							sh "docker login --username ${DOCKER_HUB_USR} --password ${DOCKER_HUB_PSW}"
+							sh 'PROFILE=none ci/verify.sh'
+							sh "ci/clean.sh"
+						}
+					}
+				}
 			}
 		}
 
 		stage("Test other configurations") {
 			when {
-				anyOf {
+				allOf {
 					branch 'master'
 					not { triggeredBy 'UpstreamCause' }
 				}
 			}
 			parallel {
 				stage("test: baseline (jdk11)") {
 					agent {
-						docker {
-							image 'adoptopenjdk/openjdk11:latest'
-							label 'data'
-							args '-v $HOME:/tmp/jenkins-home'
-						}
+						label 'data'
 					}
 					options { timeout(time: 30, unit: 'MINUTES') }
+
+					environment {
+						DOCKER_HUB = credentials('hub.docker.com-springbuildmaster')
+					}
+
 					steps {
-						sh 'rm -rf ?'
-						sh 'MAVEN_OPTS="-Duser.name=jenkins -Duser.home=/tmp/jenkins-home" ./mvnw -Pjava11 clean dependency:list test -Dsort -U -B'
+						script {
+							docker.withRegistry('', 'hub.docker.com-springbuildmaster') {
+								docker.image('adoptopenjdk/openjdk11:latest').inside('-u root -v /var/run/docker.sock:/var/run/docker.sock -v /usr/bin/docker:/usr/bin/docker -v $HOME:/tmp/jenkins-home') {
+									sh "docker login --username ${DOCKER_HUB_USR} --password ${DOCKER_HUB_PSW}"
+									sh 'PROFILE=java11 ci/verify.sh'
+									sh "ci/clean.sh"
+								}
+							}
+						}
 					}
 				}
 
-				stage("test: baseline (jdk12)") {
+				stage("test: baseline (jdk16)") {
 					agent {
-						docker {
-							image 'adoptopenjdk/openjdk12:latest'
-							label 'data'
-							args '-v $HOME:/tmp/jenkins-home'
-						}
+						label 'data'
 					}
 					options { timeout(time: 30, unit: 'MINUTES') }
+
+					environment {
+						DOCKER_HUB = credentials('hub.docker.com-springbuildmaster')
+					}
+
 					steps {
-						sh 'rm -rf ?'
-						sh 'MAVEN_OPTS="-Duser.name=jenkins -Duser.home=/tmp/jenkins-home" ./mvnw -Pjava11 clean dependency:list test -Dsort -U -B'
+						script {
+							docker.withRegistry('', 'hub.docker.com-springbuildmaster') {
+								docker.image('adoptopenjdk/openjdk16:latest').inside('-u root -v /var/run/docker.sock:/var/run/docker.sock -v /usr/bin/docker:/usr/bin/docker -v $HOME:/tmp/jenkins-home') {
+									sh "docker login --username ${DOCKER_HUB_USR} --password ${DOCKER_HUB_PSW}"
+									sh 'PROFILE=java11 ci/verify.sh'
+									sh "ci/clean.sh"
+								}
+							}
+						}
 					}
 				}
 			}
@@ -81,11 +105,7 @@ pipeline {
 				}
 			}
 			agent {
-				docker {
-					image 'adoptopenjdk/openjdk8:latest'
-					label 'data'
-					args '-v $HOME:/tmp/jenkins-home'
-				}
+				label 'data'
 			}
 			options { timeout(time: 20, unit: 'MINUTES') }
 
@@ -94,27 +114,28 @@ pipeline {
 			}
 
 			steps {
-				sh 'rm -rf ?'
-				sh 'MAVEN_OPTS="-Duser.name=jenkins -Duser.home=/tmp/jenkins-home" ./mvnw -Pci,artifactory ' +
-						'-Dartifactory.server=https://repo.spring.io ' +
-						"-Dartifactory.username=${ARTIFACTORY_USR} " +
-						"-Dartifactory.password=${ARTIFACTORY_PSW} " +
-						"-Dartifactory.staging-repository=libs-snapshot-local " +
-						"-Dartifactory.build-name=spring-data-elasticsearch " +
-						"-Dartifactory.build-number=${BUILD_NUMBER} " +
-						'-Dmaven.test.skip=true clean deploy -U -B'
+				script {
+					docker.withRegistry('', 'hub.docker.com-springbuildmaster') {
+						docker.image('adoptopenjdk/openjdk8:latest').inside('-v $HOME:/tmp/jenkins-home') {
+							sh 'MAVEN_OPTS="-Duser.name=jenkins -Duser.home=/tmp/jenkins-home" ./mvnw -Pci,artifactory -Dmaven.repo.local=/tmp/jenkins-home/.m2/spring-data-elasticsearch-non-root ' +
+									'-Dartifactory.server=https://repo.spring.io ' +
+									"-Dartifactory.username=${ARTIFACTORY_USR} " +
+									"-Dartifactory.password=${ARTIFACTORY_PSW} " +
+									"-Dartifactory.staging-repository=libs-snapshot-local " +
+									"-Dartifactory.build-name=spring-data-elasticsearch " +
+									"-Dartifactory.build-number=${BUILD_NUMBER} " +
+									'-Dmaven.test.skip=true clean deploy -U -B'
+						}
+					}
+				}
 			}
 		}
 		stage('Publish documentation') {
 			when {
 				branch 'master'
 			}
 			agent {
-				docker {
-					image 'adoptopenjdk/openjdk8:latest'
-					label 'data'
-					args '-v $HOME:/tmp/jenkins-home'
-				}
+				label 'data'
 			}
 			options { timeout(time: 20, unit: 'MINUTES') }
 
@@ -123,12 +144,18 @@ pipeline {
 			}
 
 			steps {
-				sh 'MAVEN_OPTS="-Duser.name=jenkins -Duser.home=/tmp/jenkins-home" ./mvnw -Pci,distribute ' +
-						'-Dartifactory.server=https://repo.spring.io ' +
-						"-Dartifactory.username=${ARTIFACTORY_USR} " +
-						"-Dartifactory.password=${ARTIFACTORY_PSW} " +
-						"-Dartifactory.distribution-repository=temp-private-local " +
-						'-Dmaven.test.skip=true clean deploy -U -B'
+				script {
+					docker.withRegistry('', 'hub.docker.com-springbuildmaster') {
+						docker.image('adoptopenjdk/openjdk8:latest').inside('-v $HOME:/tmp/jenkins-home') {
+							sh 'MAVEN_OPTS="-Duser.name=jenkins -Duser.home=/tmp/jenkins-home" ./mvnw -Pci,distribute -Dmaven.repo.local=/tmp/jenkins-home/.m2/spring-data-elasticsearch-non-root ' +
+									'-Dartifactory.server=https://repo.spring.io ' +
+									"-Dartifactory.username=${ARTIFACTORY_USR} " +
+									"-Dartifactory.password=${ARTIFACTORY_PSW} " +
+									"-Dartifactory.distribution-repository=temp-private-local " +
+									'-Dmaven.test.skip=true clean deploy -U -B'
+						}
+					}
+				}
 			}
 		}
 	}

README.adoc

@@ -19,7 +19,7 @@ This project is lead and maintained by the community.
 
 == Code of Conduct
 
-This project is governed by the link:CODE_OF_CONDUCT.adoc[Spring Code of Conduct]. By participating, you are expected to uphold this code of conduct. Please report unacceptable behavior to spring-code-of-conduct@pivotal.io.
+This project is governed by the https://github.com/spring-projects/.github/blob/e3cc2ff230d8f1dca06535aa6b5a4a23815861d4/CODE_OF_CONDUCT.md[Spring Code of Conduct]. By participating, you are expected to uphold this code of conduct. Please report unacceptable behavior to spring-code-of-conduct@pivotal.io.
 
 == Getting Started
 
@@ -114,7 +114,7 @@ Add the Maven dependency:
 <dependency>
   <groupId>org.springframework.data</groupId>
   <artifactId>spring-data-elasticsearch</artifactId>
-  <version>${version}.RELEASE</version>
+  <version>${version}</version>
 </dependency>
 ----
 
@@ -123,7 +123,7 @@ Add the Maven dependency:
 // Always change both files!
 **Compatibility Matrix**
 
-The compatibility between Spring Data Elasticsearch, Elasticsearch client drivers and Spring Boot versions can be found in the https://docs.spring.io/spring-data/elasticsearch/docs/3.2.0.RC3/reference/html/#preface.versions[reference documentation].
+The compatibility between Spring Data Elasticsearch, Elasticsearch client drivers and Spring Boot versions can be found in the https://docs.spring.io/spring-data/elasticsearch/docs/current/reference/html/#preface.versions[reference documentation].
 
 To use the Release candidate versions of the upcoming major version, use our Maven milestone repository and declare the appropriate dependency version:
 
@@ -149,7 +149,7 @@ If you'd rather like the latest snapshots of the upcoming major version, use our
 <dependency>
   <groupId>org.springframework.data</groupId>
   <artifactId>spring-data-elasticsearch</artifactId>
-  <version>${version}.BUILD-SNAPSHOT</version>
+  <version>${version}-SNAPSHOT</version>
 </dependency>
 
 <repository>
@@ -170,17 +170,17 @@ If you are just starting out with Spring, try one of the https://spring.io/guide
 * If you are upgrading, check out the https://docs.spring.io/spring-data/elasticsearch/docs/current/changelog.txt[changelog] for "`new and noteworthy`" features.
 * Ask a question - we monitor https://stackoverflow.com[stackoverflow.com] for questions tagged with https://stackoverflow.com/tags/spring-data[`spring-data-elasticsearch`].
 You can also chat with the community on https://gitter.im/spring-projects/spring-data[Gitter].
-* Report bugs with Spring Data for Elasticsearch at https://jira.spring.io/browse/DATAES[jira.spring.io/browse/DATAES].
+* Report bugs with Spring Data for Elasticsearch at https://github.com/spring-projects/spring-data-elasticsearch/issues[https://github.com/spring-projects/spring-data-elasticsearch/issues].
 
 == Reporting Issues
 
-Spring Data uses JIRA as issue tracking system to record bugs and feature requests. If you want to raise an issue, please follow the recommendations below:
+Spring Data uses GitHub as issue tracking system to record bugs and feature requests. If you want to raise an issue, please follow the recommendations below:
 
 * Before you log a bug, please search the
-https://jira.spring.io/browse/DATAES[issue tracker] to see if someone has already reported the problem.
-* If the issue doesn’t already exist, https://jira.spring.io/browse/DATAES[create a new issue].
-* Please provide as much information as possible with the issue report, we like to know the version of Spring Data that you are using and JVM version.
-* If you need to paste code, or include a stack trace use JIRA `{code}…{code}` escapes before and after your text.
+https://github.com/spring-projects/spring-data-elasticsearch/issues[issue tracker] to see if someone has already reported the problem.
+* If the issue doesn’t already exist, https://github.com/spring-projects/spring-data-elasticsearch/issues/new[create a new issue].
+* Please provide as much information as possible with the issue report, we like to know the version of Spring Data Elasticsearch that you are using and JVM version.
+* If you need to paste code, or include a stack trace use Markdown +++```+++ escapes before and after your text.
 * If possible try to create a test-case or project that replicates the issue. Attach a link to your code or a compressed file containing your code.
 
 == Building from Source
@@ -197,6 +197,13 @@ If you want to build with the regular `mvn` command, you will need https://maven
 
 _Also see link:CONTRIBUTING.adoc[CONTRIBUTING.adoc] if you wish to submit pull requests, and in particular please sign the https://cla.pivotal.io/sign/spring[Contributor’s Agreement] before submitting your first pull request._
 
+IMPORTANT: When contributing, please make sure an issue exists in https://github.com/spring-projects/spring-data-elasticsearch/issues[issue tracker] and comment on this issue with how you want to address it. By this we not only know that someone is working on an issue, we can also align architectural questions and possible solutions before work is invested
+. We
+so
+can prevent that much work is put into Pull Requests that have little
+or no chances of being merged.
+
+
 === Building reference documentation
 
 Building the documentation builds also the project without running tests.

TESTING.adoc

@@ -0,0 +1,41 @@
+= Testing
+
+== Unit tests
+
+Unit tests in the project are run with
+
+----
+./mvnw test
+----
+
+== Integration tests
+
+Integration tests are executed when
+----
+./mvnw verify
+----
+is run. There must be _docker_ running, as the integration tests use docker to start an Elasticsearch server.
+
+Integration tests are tests that have the Junit5 Tag `@Tag("integration-test")` on the test class. Normally this should not be set explicitly, but the annotation `@SpringIntegrationTest` should be used. This not only marks the test as integration test, but integrates an automatic setup of an Elasticsearch Testcontainer and integrate this with Spring, so
+that the required Beans can be automatically injected. Check _src/test/java/org/springframework/data/elasticsearch/JUnit5SampleRestClientBasedTests.java_ as a reference setup
+
+== Mutation testing
+
+The pom includes a plugin dependency to run mutation tests using [pitest](https://pitest.org/). These tests must be explicitly configured and run, they are not included in the normal build steps. Before pitest can run, a normal `./mvnw test` must be executed. The configuration excludes integration tests, only unit tests are considered.
+
+
+pitest can be run directly from the commandline
+----
+./mvnw org.pitest:pitest-maven:mutationCoverage
+----
+This will output an html report to _target/pit-reports/YYYYMMDDHHMI_.
+
+To speed-up repeated analysis of the same codebase set the withHistory parameter to true.
+----
+./mvnw -DwithHistory org.pitest:pitest-maven:mutationCoverage
+----
+
+The classes to test are defined either in the pom.xml or can be set on the commandline:
+----
+./mvnw -DwithHistory org.pitest:pitest-maven:mutationCoverage -DtargetClasses="org.springframework.data.elasticsearch.support.*"
+----

ci/clean.sh

@@ -0,0 +1,6 @@
+#!/bin/bash -x
+
+set -euo pipefail
+
+MAVEN_OPTS="-Duser.name=jenkins -Duser.home=/tmp/jenkins-home" \
+  ./mvnw clean -Dmaven.repo.local=/tmp/jenkins-home/.m2/spring-data-elasticsearch