Skip to content

fix(docker release): avoid duplicate occupation in docker image #8393

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Apr 19, 2025

Conversation

yanjing19989
Copy link
Contributor

Using COPY --chmod=755 instead of chmod +x prevents Docker from creating duplicate layers for the binary file, which will resolve the abnormal increase in image space usage since v3.42.0.

Although the issue only occurs in CI, I also update the local Dockerfile for consistency.

Close: #8388

…tGo#8388)

* fix(ci): modify the method of adding permissions

* fix(build): modify the method of adding permissions(to keep up with ci)
@Mmx233
Copy link
Contributor

Mmx233 commented Apr 18, 2025

Are there any images built by CI in this branch? That layer should be tiny, I'm currios about how much exactly we can save. The root cause of image size increase, I guess, is that the binary size of alist increased.

@Mmx233
Copy link
Contributor

Mmx233 commented Apr 18, 2025

Your are right, the size almost doubled because of chmod, we can save about 28MB.

@yanjing19989
Copy link
Contributor Author

This change will reduce the size of CI-built Docker images by the equivalent of the AList binary file's size. Particularly for Linux (musl) like Alpine, this will cut down image size by over 80MB.

Additionally, musl-based systems are often more space-sensitive. During testing, I noticed that the AList binary on Linux (musl) is significantly larger than on other platforms (80MB vs. 30MB). I'm unsure whether this is due to limitations in the platform's compilation toolchain.

@xhofe xhofe requested a review from Copilot April 18, 2025 03:50
Copy link
Contributor

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot wasn't able to review any files in this pull request.

Files not reviewed (2)
  • Dockerfile: Language not supported
  • Dockerfile.ci: Language not supported

@Mmx233
Copy link
Contributor

Mmx233 commented Apr 18, 2025

I haven't find out root cause yet. This should not related to platform, I have some other images use same extra layer with alpine but size didn't increase. Currently I guess it may related to whether there is x permission on binary before copy and whether docker buildkit is enabled while building.

Whatever, the fix in this PR is fantastic. I haven't try but it should able to fix the problem.

@yanjing19989
Copy link
Contributor Author

Your guess is correct. The Alist copied during the CI build lacks execute permissions, so running chmod +x would cause duplicate occupation. However, in the local multi-stage build, there is no issue with missing execute permissions, and running chmod +x does not produce any changes.

The additional Linux (musl) size issue mentioned in the previous comment is related to the executable file itself. If this can be optimized, the Docker image size could be significantly reduced once again.

Taking linux_amd64 as an example:

  • Current v3.44.0 Docker image size: 194MB
  • Docker image size after adopting this commit / local build: 106MB
  • If the Linux (musl) executable size were similar to other platforms, the Docker image size could be as low as: ~55MB

@Mmx233
Copy link
Contributor

Mmx233 commented Apr 18, 2025

Currently, musl build in CI for docker is important and it's hard to migrate to docker QEMU cross build or go toolchain cross build.

Firstly alist needs sqlite, which must enable CGO during build. So we can't use go toolchain to build directly. Secondly, alist used to be using QEMU build with docker multistage build, result in each action will run for more than an hour because of insufficiant memory. Now, after migrate to musl build, we have added more architactures to docker release work flow, so if we use QEMU again, build time will be much more higher than an hour.

As an anther workaround, it's possible to set buildkit parallel build limit to 1 to avoid using too much memory. However, using QEMU makes it almost impossible to reuse go build caches across the same workflow or multiple actions, and its emulation efficiency is significantly lower than that of the toolchain with musl, which also results in a substantial increase in CI time. So currently it's hard to migrate to other build method.

@Mmx233
Copy link
Contributor

Mmx233 commented Apr 18, 2025

Hi @xhofe, this PR LGTM and is ready to be merged. We're currently discussing further optimizations based on this PR.

Also, there is another pending PR (#7812) with an little code logic imporve — we would appreciate it if you could take a look and consider merging it as well.

@xhofe xhofe merged commit b449312 into AlistGo:main Apr 19, 2025
12 checks passed
@xhofe
Copy link
Collaborator

xhofe commented Apr 19, 2025

Nice job 👍👍👍

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

The docker image size after v3.42.0 has abnormally doubled.
3 participants