Skip to content

ci: switch to core20 for snap - #127320

Merged
Robo (deepak1556) merged 3 commits into
mainfrom
robo/fix_vscode_encrypt_snap
Mar 9, 2023
Merged

Robo (deepak1556) merged 3 commits into
mainfrom
robo/fix_vscode_encrypt_snap

Conversation

@deepak1556

@deepak1556 Robo (deepak1556) commented Jun 28, 2021 •

Copy link
Copy Markdown
Collaborator

Fixes #116690
Fixes #119436
Fixes #147166
Fixes #155094
Fixes #168427
Fixes #133856
Fixes #146890

vscode-encrypt fails to load with GLIBC_2.25 not found error.

ldd vscode-encrypt.node
  libstdc++.so.6 => /snap/core/current/usr/lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007fc887a3a000)
  libgcc_s.so.1 => /snap/core/current/lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007fc887824000)
  libpthread.so.0 => /snap/core/current/lib/x86_64-linux-gnu/libpthread.so.0 (0x00007fc887607000)
  libc.so.6 => /snap/core/current/lib/x86_64-linux-gnu/libc.so.6 (0x00007fc88723d000)
  libm.so.6 => /snap/core/current/lib/x86_64-linux-gnu/libm.so.6 (0x00007fc886f34000)
  /lib64/ld-linux-x86-64.so.2 (0x00007fc8882bc000)

libc.so.6 from ubuntu16.04 core is not sufficient for support.
Switching to core18 matches with the build environment used for the native module
https://github.com/microsoft/vscode-linux-build-agent/blob/main/bionic-x64/Dockerfile

@joaomoreno

Copy link
Copy Markdown
Contributor

Relevant: #119436

@deepak1556

Robo (deepak1556) commented Jun 28, 2021 •

Copy link
Copy Markdown
Collaborator Author

Hi Ian Johnson (@anonymouse64), I was informed that this change might be of interest to you. Thoughts ? I chose core18 to match the build environment used today.

Side note: Should I revert #119478 in favor of #118116 in this PR, since we are using base: ?

TylerLeonhardt
TylerLeonhardt previously approved these changes Jun 28, 2021

@TylerLeonhardt TylerLeonhardt left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this makes sense to me but would like to wait for someone else to sign off as well

@anonymouse64

Copy link
Copy Markdown
Contributor

If the snap that is built here is working correctly, then +1 from me, I'm happy to test a .snap file built from this PR if someone can make it available for download, I don't have the time or expertise to build VS code unfortunately. I do use the VS code snap on a daily basis however :-)

@deepak1556

Copy link
Copy Markdown
Collaborator Author

@anonymouse64

Ian Johnson (anonymouse64) commented Jul 2, 2021 •

Copy link
Copy Markdown
Contributor

unfortunately I don't think this is ready/working yet:

$ snap run code-insiders
*** stack smashing detected ***: terminated
/snap/code-insiders/x1/electron-launch: line 29: 324320 Aborted                 (core dumped) "$SNAP/usr/lib/$ARCH/gdk-pixbuf-2.0/gdk-pixbuf-query-loaders" > "$GDK_PIXBUF_MODULE_FILE"
/snap/code-insiders/x1/usr/share/code-insiders/bin/../code-insiders: relocation error: /snap/code-insiders/x1/usr/share/code-insiders/libffmpeg.so: symbol pthread_cond_destroy version GLIBC_2.3.2 not defined in file libpthread.so.0 with link time reference

did anyone manage to run this version of VS code successfully when built as a snap with core18 base ? seems like a $LD_LIBRARY_LOAD path issue where it is using the wrong glibc from the host and not from the base snap

@deepak1556

Copy link
Copy Markdown
Collaborator Author

Can confirm the crash and that the dynamic libraries are linked to the host OS, Ian Johnson (@anonymouse64) any thoughts what can cause this to happen, we are not messing with ld paths in the build.

@anonymouse64

Copy link
Copy Markdown
Contributor

Robo (@deepak1556) I will defer to Sergio Schvezov (@sergiusens) or @igorljubuncic for more help on that, I think that it might actually be that previously snapcraft was setting some paths for you, and it stopped doing this when you switched to using a base in the snapcraft.yaml.

@deepak1556

Copy link
Copy Markdown
Collaborator Author

Thanks Ian Johnson (@anonymouse64) , in case it helps we are using snapcraft:stable container to build our snaps.

@ghost

Deleted user (ghost) commented Jul 6, 2021 •

Copy link
Copy Markdown

I successfully built vscode with core20 - but you need to build it in a 20.04 container. Launching the 16.04 docker container will lead to the core dump issue as you see above. Here's the yaml that I used - but you need to edit the CI workflow to create a different snapcore container.

https://github.com/snapcore/snapcraft/blob/master/docker/Dockerfile

Similarly, if you want to use 18.04, you need an 18.04 build env.

The yaml (ignore the version, as I built it locally):

version: "1.0"
summary: Code editing. Redefined.
description: |
  Visual Studio Code is a new choice of tool that combines the
  simplicity of a code editor with what developers need for the core
  edit-build-debug cycle.

architectures:
  - build-on: amd64
    run-on: amd64

grade: stable
confinement: classic
base: core20

parts:
  code:
    plugin: dump
    source: ./code_1.54.3-1615806378_amd64.deb
    source-type: deb
    stage-packages:
      - libx11-xcb1
      - libx11-6
      - libxcomposite1
      - libxdamage1
      - libxext6
      - libxfixes3
      - libxrandr2
      - libasound2
      - libatk1.0-0
      - libatk-bridge2.0-0
      - libatspi2.0-0
      - libcairo2
      - libdrm2
      - libgbm1
      - libgdk-pixbuf2.0-0
      - libgtk-3-0
      - libnspr4
      - libnss3
      - libpango-1.0-0
      - libsecret-1-0
      - libxcb1
      - libxkbcommon0
      - libxkbfile1

apps:
  code:
    command: usr/share/code/bin/code
    common-id: code.desktop
    environment:
      DISABLE_WAYLAND: 1
      GSETTINGS_SCHEMA_DIR: usr/share/glib-2.0/schemas

  url-handler:
    command: usr/share/code/bin/code --open-url
    environment:
      DISABLE_WAYLAND: 1
      GSETTINGS_SCHEMA_DIR: usr/share/glib-2.0/schemas```

@deepak1556

Copy link
Copy Markdown
Collaborator Author

@igorljubuncic to clarify, the snap packaging is done in two steps:

First we build vscode application on a custom 18.04 container https://github.com/microsoft/vscode-linux-build-agent/blob/main/bionic-x64/Dockerfile and create a ci artifact of the build. This is the reason I chose core18 as we cannot move to 20.04 here yet.

Next in a separate pipeline we use snapcraft:stable container to run the snap commands for creating the package with the above artifact https://github.com/microsoft/vscode/blob/main/build/azure-pipelines/linux/snap-build-linux.yml

Launching the 16.04 docker container will lead to the core dump issue as you see above.

If I am reading https://github.com/snapcore/snapcraft/blob/master/docker/README.md correctly, snapcraft:stable is a 16.04 container ? Is there a 18.04 version published or should we build a custom version based on https://github.com/snapcore/snapcraft/blob/master/docker/Dockerfile ?

@ghost

Copy link
Copy Markdown

Yes, it is 16.04 - I am not aware of an existing 18.04 image. But you can spin off your own - just change the image declaration to download the right image.

@deepak1556

Copy link
Copy Markdown
Collaborator Author

Thanks for the clarification, that helps!

@deepak1556
Robo (deepak1556) force-pushed the robo/fix_vscode_encrypt_snap branch from cebe99f to 32c66f4 Compare July 13, 2021 06:11
@deepak1556

Copy link
Copy Markdown
Collaborator Author

@igorljubuncic I am trying to switch to use gnome-3-34 for core18 but looks like the extension does not work with classic confinement. What is the way forward here ?

@ghost

Copy link
Copy Markdown

One thing you can do is expand a snapcraft.yaml that has a declared extension - and then you can manually declare all the pieces that are automatically hidden behind the scenes.

snapcraft expand-extensions will look for snapcraft.yaml and expand it on the standard output, but you can also redirect the output to a file, and compare the two declarations, with and without. Please try that and see if that helps.

@anonymouse64

Copy link
Copy Markdown
Contributor

note that you can't use the gnome extension with a classic snap, and furthermore you also can't use the gnome content interface providing snaps with a classic snap as well. Has anyone tried changing this snap to just use base: core in the snapcraft.yaml? that may be enough to switch from building with the "legacy" mode snapcraft and to using new snapcraft which understands the LZO compression option and will have more improvements. It's not as nice as fully upgrading to a newer base, but it could be a stepping stone.

@deepak1556

Copy link
Copy Markdown
Collaborator Author

@igorljubuncic thanks for the pointer, expanding it shows that the extension adds some plugs to use the gnome content interface which I don't think will work with classic snaps as mentioned by Ian Johnson (@anonymouse64) ?

plugs:
  desktop:
    mount-host-font-cache: false
  gnome-3-34-1804:
    default-provider: gnome-3-34-1804
    interface: content
    target: $SNAP/gnome-platform
  gtk-3-themes:
    default-provider: gtk-common-themes
    interface: content
    target: $SNAP/data-dir/themes
  icon-themes:
    default-provider: gtk-common-themes
    interface: content
    target: $SNAP/data-dir/icons
  sound-themes:
    default-provider: gtk-common-themes
    interface: content
    target: $SNAP/data-dir/sounds

Ian Johnson (@anonymouse64) we definitely want to use base:core18 as the application is being built on ubuntu:18.04 container. The main motivation for the move was some native dependencies didn't get required glibc version #127320 (comment)

Does the above findings imply that we need strict confinement to use core18 base ? Or is there a way to use gnome-3-34 without snap interface ?

@anonymouse64

Copy link
Copy Markdown
Contributor

Robo (@deepak1556) well not necessarily, you can keep using classic and use a core18 base, but the gnome extension is our preferred way of setting up the environment variables and other such things so that most strictly confined applications "just work" out of the box without special fiddling in the YAML. You can of course setup everything that is setup by the extension manually, and without the content interface connections. The content interface simply shares the files from these other snaps with your snap so that your snap does not need to ship those files and there is less duplication across different graphical gnome apps.

There are a few options to move forward if you determine that you do indeed need the libraries and files from these gnome content snaps.

1.) You can simply copy the files from these other snaps into your own snap and put them in equivalent locations, for example the files that gnome-3-34-1804 content plug shares are put at $SNAP/gnome-platform - you can add a part to your snapcraft.yaml which copies these files into $SNAPCRAFT_PART_INSTALL/gnome-platform and it will work the same as if you had the content interface connected, albeit your snap will be slightly larger. Another disadvantage of this is that you need to handle updates to these other snaps yourself when you build these snaps, i.e. normally if there is an update/fix to the gnome-3-34-1804 snap, it can be pushed out and dependent strict applications can get it immediately without needing to rebuild their snap like gnome-calculator for example, but here since you are copying the files into your snap you need to rebuild the snap to benefit from the updates. this may not be an issue for VS code, since updates to the snap seem to be relatively recent judging by at least the updates I get on my machine...
2.) You can in fact use files from other snaps in a classic snap like with plugs for a strict snap, by doing the mounts that correspond to the implementation of the content interface manually, i.e. mount --bind /snap/gnome-3-34-1804/current/gnome-platform $SNAP/gnome-platform will do the same thing as the content interface plug mentioned above, your snap just needs to make sure that gnome-3-34-1804 is installed, which it can actually do since it is a classic snap and not strictly confined. The issue with this is that you need to be careful to make sure that snaps you depend on are installed, and that you (maybe) monitor for changes to the source bind mounts, because for example if you do the bind mount as I mentioned above in a shell script that is part of the launch of VS code, then a refresh happens to gnome-3-34-1804, your snap will still have the previous revision mounted, since bind mounts don't follow new symlink targets IIRC. This may not be a big deal since your application will just continue to use the old revisions of the dependent snaps until the user reboots their machine or logs out or something like that and you perform the mounts again.
3. We (snapd developers) are aware of this limitation and have it generally as a TODO to make content interfaces work with classic snaps the same way they do with strict snaps, and the biggest part of that work was actually already done to support parallel instances of classic snaps, but there is still a bit of work left to do to fully enable this use-case. So you could just wait until we enable the content interface for classic snaps, but it's not on our immediate roadmap so it could be a few months (or maybe even a year or two) before this is available to use, but the VS code snap is not the only one that suffers from this issue, so the more snaps that are suffering from this situation the more likely we are to fix it sooner I suppose.

I would recommend experimenting a bit with 1 as a start

@deepak1556

Copy link
Copy Markdown
Collaborator Author

Thanks Ian Johnson (@anonymouse64) , I will give 1) a try looks like it might get this PR moving again.

@deepak1556
Robo (deepak1556) force-pushed the robo/fix_vscode_encrypt_snap branch 6 times, most recently from 58f1b0f to 5b064cd Compare July 20, 2021 05:22
@dverbru

Copy link
Copy Markdown

This blocks #168427.

@deepak1556 Robo (deepak1556) changed the title ci: switch to core18 for snap ci: switch to core22 for snap Feb 1, 2023
@deepak1556
Robo (deepak1556) force-pushed the robo/fix_vscode_encrypt_snap branch from a468471 to e996283 Compare February 22, 2023 14:42
@deepak1556 Robo (deepak1556) changed the title ci: switch to core22 for snap ci: switch to core20 for snap Feb 22, 2023
@deepak1556

Robo (deepak1556) commented Feb 23, 2023 •

Copy link
Copy Markdown
Collaborator Author

This iteration I tried to revamp this PR and update our snap base to core20, although overall build is looking good there are a couple of issues that needs to be addressed before getting to the finish line. Following test was made on Fedora 36

  • dlopen calls from the binary end up loading libraries from the system and not the ones bundled with the snap
    • Some example failures, first when trying to load EGL for software rendering, second when trying to load curl for uploading crash reports. There are more scenarios in the runtime, just these two are caught on startup,
Failed to load EGL library: libEGL.so.1: /snap/core20/current/lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.33' not found (required by /lib64/libEGL.so.1)

 dlopen:libcurl.so: cannot open shared object file: No such file or directory
[0223/173421.249105:ERROR:http_transport_libcurl.cc(118)] dlopen:libcurl-gnutls.so.4: cannot open shared object file: No such file or directory
[0223/173421.249113:ERROR:http_transport_libcurl.cc(118)] dlopen:libcurl-nss.so.4: cannot open shared object file: No such file or directory
[0223/173421.249118:ERROR:http_transport_libcurl.cc(118)] dlopen:/snap/core20/current/lib/x86_64-linux-gnu/libc.so.6: version 'GLIBC_2.33' not found (required by /lib64/libcurl.so.4)
  • Library is available at following locations inside /snap/code-insiders/current
$ cd /snap/code-insiders/current/
[parallels@fedora current]$ find . -name libEGL*
./gnome-platform/usr/lib/x86_64-linux-gnu/libEGL.so
./gnome-platform/usr/lib/x86_64-linux-gnu/libEGL.so.1
./gnome-platform/usr/lib/x86_64-linux-gnu/libEGL.so.1.1.0
./gnome-platform/usr/lib/x86_64-linux-gnu/libEGL_mesa.so.0
./gnome-platform/usr/lib/x86_64-linux-gnu/libEGL_mesa.so.0.0.0
./usr/share/code-insiders/libEGL.so

[parallels@fedora current]$ find . -name libcurl*
./usr/lib/x86_64-linux-gnu/libcurl-gnutls.so.3
./usr/lib/x86_64-linux-gnu/libcurl-gnutls.so.4
./usr/lib/x86_64-linux-gnu/libcurl-gnutls.so.4.6.0
./usr/lib/x86_64-linux-gnu/libcurl-nss.so.3
./usr/lib/x86_64-linux-gnu/libcurl-nss.so.4
./usr/lib/x86_64-linux-gnu/libcurl-nss.so.4.6.0
./usr/lib/x86_64-linux-gnu/libcurl.so.4
./usr/lib/x86_64-linux-gnu/libcurl.so.4.6.0
  • Reading up on how snapcraft adjusts RPATH for classic confinement to be able to load the correct libraries https://snapcraft.io/docs/classic-confinement, I have added a step in the prime stage to output the final adjusted RPATH of the binary and this is the current output
+ patchelf --print-rpath /__w/1/s/.build/linux/snap/x64/code-insiders-x64/prime/usr/share/code-insiders/code-insiders
 $ORIGIN/../../lib/x86_64-linux-gnu:$ORIGIN:/snap/core20/current/lib/x86_64-linux-gnu

@deepak1556
Robo (deepak1556) force-pushed the robo/fix_vscode_encrypt_snap branch from e996283 to 0346e2b Compare March 7, 2023 04:15
@deepak1556 Robo (deepak1556) added this to the March 2023 milestone Mar 7, 2023
@deepak1556
Robo (deepak1556) marked this pull request as ready for review March 8, 2023 02:46
@deepak1556

Robo (deepak1556) commented Mar 8, 2023 •

Copy link
Copy Markdown
Collaborator Author

João Moreno (@joaomoreno) this is now good to go 🚀

Build for verification: Insiders snap x64

@joaomoreno João Moreno (joaomoreno) left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is great Robo (@deepak1556), thanks for all the hard work! After my round of testing I didn't hit any blockers, so let's go ahead with this. 💪

@joaomoreno João Moreno (joaomoreno) added the snap Issues related to the snap package label Mar 8, 2023
@deepak1556
Robo (deepak1556) merged commit 502ac5b into main Mar 9, 2023
@deepak1556
Robo (deepak1556) deleted the robo/fix_vscode_encrypt_snap branch March 9, 2023 02:14
@github-actions github-actions Bot locked and limited conversation to collaborators Apr 23, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

snap Issues related to the snap package

Projects

None yet

7 participants

⌂ 换网址