Project

General

Profile

Feature #4147

Improve Gradle build for offline use

Added by Hynek Cihlar almost 5 years ago. Updated over 4 years ago.

Status:
Closed
Priority:
Normal
Assignee:
Target version:
-
Start date:
Due date:
% Done:

100%

billable:
No
vendor_id:
GCD

build.gradle (30.5 KB) Hynek Cihlar, 07/17/2019 04:59 PM

build.gradle (31 KB) Hynek Cihlar, 07/17/2019 05:35 PM

History

#1 Updated by Hynek Cihlar almost 5 years ago

  • Subject changed from Lock dependencies resolved with Gradle to Improve Gradle build for offline use

#2 Updated by Hynek Cihlar almost 5 years ago

Current issues:

  • Gradle wrapper or a Gradle distribution must must be downloaded before Gradle build can be run.
  • Current approach of copying the Gradle cache doesn't seem to handle the cases when a modul version is changed by Gradle during dependency resolution.
  • Current approach of copying the Gradle cache may break when the Gradle cache format changes.
The solution:
  • Download the Gradle wrapper binaries and put them in the dependency zip archive, change the Gradle wrapper url to point to local file system (local-repo). Modify distributionUrl in gradle-wrapper.properties.
  • In the build script initialize the configurations conditionally either using the module name dependency syntax or directly from files on the local file system depending whether in online mode or not. Something like the below.
    dependencies {
        if (gradle.startParameter.isOffline()) {
            fwdLocalLibs fileTree(dir: localLibs)
        } else {
            fwdClient group: 'com.twelvemonkeys.common', name: 'common-image', version: '3.1.2'
        }
    }
    
  • Create the local lib dir by copying the resolved files directly from the configurations.
    task refreshLocalLibs(dependsOn: 'deleteLocalLibs') << {
        ['fwdAllCompile', 'fwdAllRuntime'].each { cfg ->
            copy {
                from configurations.getByName(cfg).files
                into "${localLibs}/${cfg}" 
            }
        }
    }
    

#3 Updated by Hynek Cihlar almost 5 years ago

The ideas above are implemented in the attached build.gradle. The changes are based on the FWD version 3.2.0. The changes were tested on Linux and Windows with and without internet connectivity, with fresh Gradle installs of versions 3.5 and 5.5.

#4 Updated by Hynek Cihlar almost 5 years ago

Another version of build.gradle attached. I added the task fetchGradle which will fetch the Gradle distribution to the lib dir. This file will be used as the distribution url for the Gradle wrapper script during offline build.

To use the distribution on a fresh system without internet connectivity set the property distributionUrl in gradle/wrapper/gradle-wrapper.properties to ../../lib/gradle-3.0-bin.zip.

#5 Updated by Hynek Cihlar almost 5 years ago

Hynek Cihlar wrote:

Another version of build.gradle attached.

And the file.

#6 Updated by Greg Shah almost 5 years ago

I'm good with the changes.

#7 Updated by Hynek Cihlar almost 5 years ago

I created task branch 4147a from version FWD 3.2.0 (trunk revision 11281). Checked in "Fix to allow locale definition to work on newer Linux versions (e.g. Ubuntu 18.04 and later). These changes also work on older Linux versions. Refs #3960." from task branch 4069a (revision 11333).

Also checked in the attached build.gradle and increased version in version.properties to 3.2.1.

With the changes during the release process, instead of calling createLocalRepo Gradle task call the tasks fetchDependencies and fetchGradle in this order. The dependencies will be fetched to the lib dir, zip this dir and use this as the dependencies archive.

To build on an offline system, set the property distributionUrl in gradle/wrapper/gradle-wrapper.properties to ../../lib/gradle-3.0-bin.zip and use the --offline Gradle parameter.

I'll update the wiki doc. Btw., shouldn't we document the release process on the wiki, too?

#8 Updated by Hynek Cihlar almost 5 years ago

With the new lib layout some jar files will be duplicated (cca 100 MB of duplicated size). Unfortunately zip doesn't allow a deduplication strategy, but other archive formats do. See https://stackoverflow.com/questions/27457326/compressing-a-folder-with-many-duplicated-files.

#9 Updated by Greg Shah over 4 years ago

I've merged the build.gradle changes into 4069a revision 11434.

Hynek: Please review.

#10 Updated by Greg Shah over 4 years ago

With the changes during the release process, instead of calling createLocalRepo Gradle task call the tasks fetchDependencies and fetchGradle in this order. The dependencies will be fetched to the lib dir, zip this dir and use this as the dependencies archive.

I'm trying to release v3.2.1 but the normal ./gradlew core does not populate lib/ with anything, that directory is empty. How do I capture it?

#11 Updated by Hynek Cihlar over 4 years ago

Greg Shah wrote:

With the changes during the release process, instead of calling createLocalRepo Gradle task call the tasks fetchDependencies and fetchGradle in this order. The dependencies will be fetched to the lib dir, zip this dir and use this as the dependencies archive.

I'm trying to release v3.2.1 but the normal ./gradlew core does not populate lib/ with anything, that directory is empty. How do I capture it?

Use the tasks fetchDependencies and fetchGradle to populate the lib directory.

#12 Updated by Hynek Cihlar over 4 years ago

Greg Shah wrote:

I've merged the build.gradle changes into 4069a revision 11434.

Hynek: Please review.

The merge is OK.

#13 Updated by Greg Shah over 4 years ago

  • % Done changed from 0 to 100
  • Status changed from New to Closed
  • Assignee set to Hynek Cihlar

I've released FWD v3.2.1 which is based on branch 4147a. The build.gradle change will merge to trunk with 4069a.

#15 Updated by Greg Shah over 4 years ago

4147a was archived as merged.

Also available in: Atom PDF