Arvados

Update: I have worked on the versioning for the package, using the common $ARVADOS_BUILDING_VERSION we use everywhere for this , it is ready. I am now working on the publishing part which is a bit delayed because i'm not totally understanding how to do it but working on it following this guide https://central.sonatype.org/pages/gradle.html , to be able to push our released artifacts I had to open a ticket on Sonatype so they will allow us to push this.
This is the ticket https://issues.sonatype.org/browse/OSSRH-47731?page=com.atlassian.jira.plugin.system.issuetabpanels%3Aall-tabpanel according to this guide to publish https://central.sonatype.org/pages/ossrh-guide.html

So I guess now we wait for someone to process the ticket so we can publish?

@Peter it has just been processed, here is the info:

Joel Orlina added a comment - 1 hour ago
Configuration has been prepared, now you can:
Deploy snapshot artifacts into repository https://oss.sonatype.org/content/repositories/snapshots
Deploy release artifacts into the staging repository https://oss.sonatype.org/service/local/staging/deploy/maven2
Promote staged artifacts into repository 'Releases'
Download snapshot and release artifacts from group https://oss.sonatype.org/content/groups/public
Download snapshot, release and staged artifacts from staging group https://oss.sonatype.org/content/groups/staging

I need now to make the gradle publishing work :)

Update: I have pushed changes to the branch for test-in-docker.sh and build.gradle to be able to build and publish jar artifact to oss.sonatype.org
running ./test-in-docker.sh 0.0.9 just to test and upload a dummy version to test the upload.

Task :compileJava UP-TO-DATE
Task :processResources UP-TO-DATE
Task :classes UP-TO-DATE
Task :jar

Task :javadoc

/Users/fmonserrat/veritas/arvados/sdk/java-v2/src/main/java/org/arvados/client/api/client/CountingFileRequestBody.java:23: warning - Tag @link: reference not found:
/Users/fmonserrat/veritas/arvados/sdk/java-v2/src/main/java/org/arvados/client/utils/FileSplit.java:20: warning - Tag @link: reference not found:
/Users/fmonserrat/veritas/arvados/sdk/java-v2/src/main/java/org/arvados/client/api/client/CountingFileRequestBody.java:23: warning - Tag @link: reference not found:
/Users/fmonserrat/veritas/arvados/sdk/java-v2/src/main/java/org/arvados/client/utils/FileSplit.java:20: warning - Tag @link: reference not found:
/Users/fmonserrat/veritas/arvados/sdk/java-v2/src/main/java/org/arvados/client/api/client/CountingFileRequestBody.java:23: warning - Tag @link: reference not found:
/Users/fmonserrat/veritas/arvados/sdk/java-v2/src/main/java/org/arvados/client/utils/FileSplit.java:20: warning - Tag @link: reference not found:
6 warnings

Task :javadocJar
Task :sourcesJar
Task :signArchives
Task :uploadArchives

BUILD SUCCESSFUL in 4m 49s
8 actionable tasks: 6 executed, 2 up-to-date

I still need to work on setup this in Jenkins and how to pass to Jenkins the gpg key to sign the artifact and other config to push it to sonatype

I have configured a jenkins job to run the tests , build and publish the package to sonatype https://ci.curoverse.com/job/build-java-sdk/
I see 2 tests failing
18:26:31 ---- Test results ----
18:26:31 FAILURE (46 tests, 44 successes, 2 failures, 0 skipped)

18:25:50 org.arvados.client.logic.collection.ManifestFactoryTest > manifestIsCreatedAsExpected FAILED
18:25:50 java.io.FileNotFoundException at ManifestFactoryTest.java:25
18:26:16 org.arvados.client.facade.ArvadosFacadeTest > uploadIsPerformedSuccessfullyUsingDiskOnlyKeepServices FAILED
18:26:16 java.lang.NullPointerException at ArvadosFacadeTest.java:67
The same happens in my local env and another VM I used to test.

If i skip the test and just run the jar install uploadArchives it works and uploads the artifacts to OSS.SONATYPE

I skipped the failing tests to get unblocked for the time being.

https://ci.curoverse.com/job/build-java-sdk/12/console

2:01:17 cp: cannot create regular file '/home/ci-jenkins/.jenkins-slave/workspace/build-java-sdk/sdk/java-v2/gradle.properties': Permission denied

I'm trying to come up with a minimum gradle project that will pull in this jar file:

apply plugin: 'java-library'
apply plugin: 'maven'

repositories {
    mavenCentral()
}

dependencies {
    api 'org.arvados:arvados-java-sdk:0.0.9'
}

But it is failing:

> Could not resolve all files for configuration ':compileClasspath'.
   > Could not find org.arvados:arvados-java-sdk:0.0.9.
     Searched in the following locations:
       - https://repo.maven.apache.org/maven2/org/arvados/arvados-java-sdk/0.0.9/arvados-java-sdk-0.0.9.pom
       - https://repo.maven.apache.org/maven2/org/arvados/arvados-java-sdk/0.0.9/arvados-java-sdk-0.0.9.jar
     Required by:
         project :

Maybe we need to do a "release" first ?

Also I noticed that the uploaded screenshot includes "arvados-java-sdk-0.0.9-javadoc.jar" and "arvados-java-sdk-0.0.9-sources.jar" but not "arvados-java-sdk-0.0.9.jar" so I'm not sure the jar with the compiled classes is being uploaded?

Update: @Peter, the java jar is there i checked it and it looked good in staging release.
You are right about the release part that needs to be done but the signing is failing when it is evaluated
Event: Failed: Signature Validation
I'm trying to fix it by using another key https://ci.curoverse.com/job/build-java-sdk/14/console

Fernando Monserrat wrote:

Update: @Peter, the java jar is there i checked it and it looked good in staging release.
You are right about the release part that needs to be done but the signing is failing when it is evaluated
Event: Failed: Signature Validation
I'm trying to fix it by using another key https://ci.curoverse.com/job/build-java-sdk/14/console

Ok, can you do a release as 0.0.9 version (or whatever) so I can test the "how to use the SDK" instructions are able to download the jar file, then we'll put out a stable release?

I could do a release for version 0.1.0 on sonatype and I can see it published https://oss.sonatype.org/content/groups/public/org/arvados/arvados-java-sdk/

This is how to move the artifact from staging repo to release
Close and Drop or Release Your Staging Repository
After your deployment the repository will be in an Open status. You can evaluate the deployed components in the repository using the Contents tab. If you believe everything is correct you, can press the Close button above the list. This will trigger the evaluations of the components against the requirements.

Closing will fail if your components do not meet the requirements. If this happens, you can press Drop and the staging repository will be deleted. This allows you to correct any problems with the components and the deployment process and re-run the deployment. Details are available in the Activity tab below the list by selecting. Press on the individual steps for further details.

Once you have successfully closed the staging repository, you can release it by pressing the Release button. This will move the components into the release repository of OSSRH where it will be synced to the Central Repository.

FYI, notes from Sonatype : Central sync is activated for org.arvados. After you successfully release, your component will be published to Central, typically within 10 minutes, though updates to search.maven.org can take up to two hours.

I've committed a change to split the uploadArchives to sonatype 30298819bc92b5ec90c8544bee7e102cb64e888d, so we'll have a job to test and build an artifact locally only and another to run the entire test, jar build and upload to release it