Skip to content
This repository was archived by the owner on Oct 29, 2026. It is now read-only.

fix(hermetic-build): obtain gapic-generator-java locally on release branch - #2023

Merged
diegomarquezp merged 9 commits into
mainfrom
ggj-local-release-branch
Sep 21, 2023
Merged

diegomarquezp merged 9 commits into
mainfrom
ggj-local-release-branch

Conversation

@diegomarquezp

Copy link
Copy Markdown
Contributor

This prevents the failures from showcase tests on the release-please branch since it tries to fetch a non-published version (2.26.0). The fix is to check whether we are on a release please branch as an additional condition to fetch the generator jar and pom locally.

@diegomarquezp
diegomarquezp requested a review from a team September 21, 2023 15:19
@product-auto-label product-auto-label Bot added the size: s Pull request size is small. label Sep 21, 2023
Comment thread library_generation/utilities.sh Outdated
if [[ "${gapic_generator_version}" == *"-SNAPSHOT" ]]; then
# release please branches will point to a non-snapshot that hasn't been
# published. We want to get the generator pom locally in this case
if [[ "${gapic_generator_version}" == *"-SNAPSHOT" ]] || [[ "$(is_release_branch)" == "true" ]]; then

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we always try to download from local maven repo first regardless of the name, and fall back to maven central if not exist? We are going to have multiple release branches in the future and this would become hard to maintain.
Similarly, we would have the same problem in the main generation script, can we try to fix it there as well @JoeWang1127 ?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks for the quick review!
I modified the generator download (pom & jar) to default to a local fetch before trying the external source

Comment thread library_generation/utilities.sh Outdated
# copy a SNAPSHOT version from maven local repository.
copy_from "$HOME/.m2/repository/com/google/api/gapic-generator-java-pom-parent/${gapic_generator_version}/gapic-generator-java-pom-parent-${gapic_generator_version}.pom" \
# first, try to fetch the generator pom locally
bash -c "source ${util_script_dir}/utilities.sh && copy_from \"$HOME/.m2/repository/com/google/api/gapic-generator-java-pom-parent/${gapic_generator_version}/gapic-generator-java-pom-parent-${gapic_generator_version}.pom\" \

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do we need source command here?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added this to load the functions into the secondary shell. This is all with the intent to capture the exit code in case the local search doesn't work.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since we always try to copy files locally before downloading from maven central, I think we can modify copy_from not failing but return a value to indicate whether copy is success or not.

Also, we need to modify the error message in download_fail to indicate the file is not found neither locally or in maven central should the download fail.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for pointing this out, makes sense. I modified copy_from to return (echo) a value instead. I added a message to indicate an external repo attempt will be made in case it wasn't found locally. I also created a more generic function for the pom parent and the jar.

@product-auto-label product-auto-label Bot added size: m Pull request size is medium. and removed size: s Pull request size is small. labels Sep 21, 2023
Comment thread library_generation/utilities.sh Outdated
local save_as=$2
cp "${local_repo}" "${save_as}" || \
download_fail "${save_as}" "maven local"
cp "${local_repo}" "${save_as}" && echo "false" || echo "true"

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what does this line mean?

It seems to echo true if cp failed.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added a comment on top of the function. I will modify the comment for it to have a better explanation

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

also added a variable to hold the result in order to improve readability

Comment thread library_generation/utilities.sh Outdated
}

# copies the specified file in $1 to $2
# will return "false" if the copy was successful, otherwise true (to indicate

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It maybe unintuitive to indicate a successful copy as false, though it's a minor issue.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I inverted the logic and changed variable names accordingly

if [[ "${local_fetch_successful}" == "false" ]];then
# download gapic-generator-java artifact from Google maven central mirror if not
# found locally
>&2 echo "${artifact} not found locally. Attempting a download from Maven Central"

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we add logs for successful local fetch or Maven central download? Otherwise it may be misleading that this is the only log we see when we are downloading from Maven Central.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks, I added a couple success messages for both local and remote fetch

@sonarqubecloud

Copy link
Copy Markdown

[gapic-generator-java-root] Kudos, SonarCloud Quality Gate passed!    Quality Gate passed

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 0 Code Smells

No Coverage information No Coverage information
No Duplication information No Duplication information

@sonarqubecloud

Copy link
Copy Markdown

[java_showcase_integration_tests] Kudos, SonarCloud Quality Gate passed!    Quality Gate passed

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 0 Code Smells

No Coverage information No Coverage information
No Duplication information No Duplication information

@sonarqubecloud

Copy link
Copy Markdown

[java_showcase_unit_tests] Kudos, SonarCloud Quality Gate passed!    Quality Gate passed

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 0 Code Smells

No Coverage information No Coverage information
No Duplication information No Duplication information

@diegomarquezp
diegomarquezp merged commit f3088d5 into main Sep 21, 2023
@diegomarquezp
diegomarquezp deleted the ggj-local-release-branch branch September 21, 2023 20:44
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

size: m Pull request size is medium.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

⌂ 换网址