Allow linker to perform deadcode elimination for program using Cobra

[aarzilli]

Fixes #2015

Cobra, in its default configuration, will execute a template to generate help, usage and version outputs. Text/template execution calls MethodByName and MethodByName disables dead code elimination in the Go linker, therefore all programs that make use of cobra will be linked with dead code elimination disabled, even if they end up replacing the default usage, help and version formatters with a custom function and no actual text/template evaluations are ever made at runtime.

Dead code elimination in the linker helps reduce disk space and memory utilization of programs. For example, for the simple example program used by TestDeadcodeElimination 40% of the final executable size is dead code. For a more realistic example, 12% of the size of Delve's executable is deadcode.

This PR changes Cobra so that, in its default configuration, it does not automatically inhibit deadcode elimination by:

1. changing Cobra's default behavior to emit output for usage and help using simple Go functions instead of template execution
2. quarantining all calls to template execution into SetUsageTemplate, SetHelpTemplate and SetVersionTemplate so that the linker can statically determine if they are reachable

See https://github.com/aarzilli/whydeadcode to help find why dead-code-elimination may not work as well as it could in your project.

[CLAassistant]

All committers have signed the CLA.

[github-actions]

This PR exceeds the recommended size of 200 lines. Please make sure you are NOT addressing multiple issues with one PR. Note this PR might be rejected due to its size.

[aarzilli]

The CLA thing isn't working for some reason.

[github-actions]

This PR exceeds the recommended size of 200 lines. Please make sure you are NOT addressing multiple issues with one PR. Note this PR might be rejected due to its size.

[marckhouzam]

Thanks @aarzilli !
I like the idea of optimizing things.
This will require some thought so I can understand clearly the behaviour change. Thanks for adding a test it should help with this.

[aarzilli]

Unless I made a mistake there shouldn't be any behavior changes (as in, observable from the outside). Happy to explain anything about this if needed.

[aarzilli]

Ping?

[aarzilli]

Ping?

[github-actions]

This PR exceeds the recommended size of 200 lines. Please make sure you are NOT addressing multiple issues with one PR. Note this PR might be rejected due to its size.

[github-actions]

This PR exceeds the recommended size of 200 lines. Please make sure you are NOT addressing multiple issues with one PR. Note this PR might be rejected due to its size.

[github-actions]

This PR exceeds the recommended size of 200 lines. Please make sure you are NOT addressing multiple issues with one PR. Note this PR might be rejected due to its size.

[github-actions]

This PR exceeds the recommended size of 200 lines. Please make sure you are NOT addressing multiple issues with one PR. Note this PR might be rejected due to its size.

[github-actions]

This PR exceeds the recommended size of 200 lines. Please make sure you are NOT addressing multiple issues with one PR. Note this PR might be rejected due to its size.

[marckhouzam]

@aarzilli I apologize for such a long delay, but I had misunderstood the value of this PR. I now realize that it may help all programs using Cobra, so I'm very interested.

I'm trying to convince myself that that this change actually has an impact.
Would you be able to explain how I can see the difference in a program that has dead code, before and after this PR?

Here is what I tried.

• I used the program you have added to the tests in this PR
• I added an "Unused()" function that I don't call in the program.
• I ran go build -o myprog . (using cobra 1.8.1 without your PR)
• I ran go tool nm ./myprog | grep Unused

I would have expected to see the Unused function but I don't.
I used go version go1.22.8 darwin/arm64

Could you clarify what I should expect?

This is the program:

package main

import (
	"fmt"
	"os"

	"github.com/spf13/cobra"
)

func Unused() {
	fmt.Println("not called")
}

var rootCmd = &cobra.Command{
	Version: "1.0",
	Use:     "example_program",
	Short:   "example_program - test fixture to check that deadcode elimination is allowed",
	Run: func(cmd *cobra.Command, args []string) {
		fmt.Println("hello world")
	},
	Aliases: []string{"alias1", "alias2"},
	Example: "stringer --help",
}

func main() {
	if err := rootCmd.Execute(); err != nil {
		fmt.Fprintf(os.Stderr, "Whoops. There was an error while executing your CLI '%s'", err)
		os.Exit(1)
	}
}

[aarzilli]

One easy way to see the impact of this change is to compile your example program with and without this PR:

$ ls -lh
total 8.4M
-rwxr-xr-x. 1 a a 5.2M Oct 14 14:45 example.withoutpr*
-rwxr-xr-x. 1 a a 3.2M Oct 14 14:45 example.withpr*
-rw-r--r--. 1 a a  596 Oct 14 14:44 main.go

This is an extreme example, where the size of the executable is reduced by 38%, but reductions of 10% are realistic.
I used go1.23 but you should get very similar results with go1.22.8.

As to your question, saying that deadcode elimination gets disabled is incorrect: in reality it continues to function but in a reduced capacity. Specifically if reflect.Value.MethodByName or reflect.Value.Method are reachable it can not always prove that exported methods are unreachable.

Your Unused function is not an exported method so, unless it is called by an exported method, it can always be deadcode eliminated.

If you want to see the Unused function make it all the way to the executable you have to do something like this:

package main

import (
	"fmt"
	"os"

	"github.com/spf13/cobra"
)

type Astruct struct {
	n int
}

//go:noinline
func Unused() {
	fmt.Println("not called")
}

//go:noinline
func (a *Astruct) Unused() {
	fmt.Println("not called", a.n)
	Unused()
}

//go:noinline
func (a *Astruct) Used() {
	fmt.Println("used", a.n)
}

var rootCmd = &cobra.Command{
	Version: "1.0",
	Use:     "example_program",
	Short:   "example_program - test fixture to check that deadcode elimination is allowed",
	Run: func(cmd *cobra.Command, args []string) {
		fmt.Println("hello world")
	},
	Aliases: []string{"alias1", "alias2"},
	Example: "stringer --help",
}

func main() {
	if err := rootCmd.Execute(); err != nil {
		fmt.Fprintf(os.Stderr, "Whoops. There was an error while executing your CLI '%s'", err)
		os.Exit(1)
	}
	var a Astruct
	fmt.Println(a)
	a.Used()
}

All of the noinline directives are needed because small functions like this would be removed by the inliner and the fmt.Println call is needed to make the Astruct type itself reachable (if all calls to the methods of Astruct are static the linker can still prove that Unused is unreachable).

With these changes you get:

$ go build main2.go && go tool nm ./main2 | grep 'Unused'
  589c00 T main.(*Astruct).Unused
  589ba0 T main.Unused
  765570 D runtime.adviseUnused
  4179e0 T runtime.sysUnusedOS

without the PR and:

$ go build main2.go && go tool nm ./main2 | grep 'Unused'
  602564 D runtime.adviseUnused
  4144a0 T runtime.sysUnusedOS

with the PR.

[marckhouzam]

Thanks for the explanation @aarzilli, I can now see the benefits.

So, with this PR, programs that don't set new templates (usage, help or version) will be able to do dead code elimination fully.

What about programs that do modify those templates? If they want to get full usage of dead code elimination they should convert their template use to a go function like you have done, IIUC? My next step for this review is to try to override the help/usage/version in that fashion and see if it works as expected.

I believe that using SetHelpFunc() and SetUsageFunc() will allow programs to do this, but I don't think we have a way to do this for the version template. But it is ok, it is still better than not being able to do it at all. Once this is merged, we can discuss adding a SetVersionFunc().

[marckhouzam]

This looks great.
I still want to do some testing, but I don't expect any big surprises.
Here are some comments for the PR.
The PR also needs to be rebased.

Thanks again for your patience, I think we can get this in soon.

[marckhouzam]

What do you think about updating the documentation in the three sections starting here https://github.com/spf13/cobra/blob/main/site/content/user_guide.md#defining-your-own-help to explain the side-effect of overriding the template, and therefore that it is recommended to set a function instead?

[github-actions]

This PR exceeds the recommended size of 200 lines. Please make sure you are NOT addressing multiple issues with one PR. Note this PR might be rejected due to its size.

[aarzilli]

Thanks for the explanation @aarzilli, I can now see the benefits.

So, with this PR, programs that don't set new templates (usage, help or version) will be able to do dead code elimination fully.

What about programs that do modify those templates? If they want to get full usage of dead code elimination they should convert their template use to a go function like you have done, IIUC?

Yes, this is correct.

I believe that using SetHelpFunc() and SetUsageFunc() will allow programs to do this, but I don't think we have a way to do this for the version template. But it is ok, it is still better than not being able to do it at all. Once this is merged, we can discuss adding a SetVersionFunc().

It imagine should be trivial.

[github-actions]

This PR exceeds the recommended size of 200 lines. Please make sure you are NOT addressing multiple issues with one PR. Note this PR might be rejected due to its size.

[marckhouzam]

@aarzilli thank you for your patience. This took a very long time but you remained very responsive. Your nice attitude (and great technical expertise) made it possible to get this merged. This should benefit many programs!

P.S. I wouldn't be against adding a section to the Cobra documentation teaching people what they should do to try to get dead-code-elimination working. Although this is more of a general Go concept, it might help people understand it if they have a small section in the Cobra docs. But I leave that up to you, if you still have energy after this looooong review.

[marckhouzam]

This is a very good improvement. @marckhouzam, could you please make a new release of cobra?

Nice to see immediate interest 😄 .
I'll work on a release this week.

[aarzilli]

Thanks for getting this merged!

[sylr]

@jpmcb Could we get a cobra release soon to be able to enjoy this change ?

[artem-anisimov-0x7f]

Is there a chance to have v1.9.0 released soon to make this fix available?

[marckhouzam]

🎉 https://github.com/spf13/cobra/releases/tag/v1.9.0
Thanks @aarzilli. The dead code elimination improvement will be valuable to many