Hi @tworcester thanks for your contribution. After taking a look at this i think we'll probably want to take a slightly different approach that is more configurable and still maintains backwards compatibility. I'm thinking it will look something like:

1. use the existing endpoint field instead of endpointDomain
2. default the existing endpoint field to empty string
3. update the template to set default values for $endpoint based on whether the provider field is minio or aws and have those defaults match the current implementation
4. if the endpoint field is specified use that value instead.

The endpointDomain option is more flexible than what exists today, but the reality is we cannot necessarily assume the s3. part of the endpoint either as there are also potentially other endpoints such as s3-fips..ors3.dual-stack..`.

You're welcome to take a stab at this if you like, otherwise we'll try to prioritize this for the next release after the update to 17.9.2 that is currently in progress.

Took a crack at it! Let me know what you think. @zachariahmiller

Took a crack at it! Let me know what you think. @zachariahmiller

@tworcester At first glance i think this looks good, but I'm also realizing now it might not be enough for your use case without additional changes.

As it is today, for AWS we do not have to specify the endpoint in any of the sections besides backups. However, for your scenario where the s3 endpoint is mapped to a different DNS entry i don't think it will "just work" as it does right now for the connection and registry sections. I'm thinking that regionendpoint in registry and host or endpoint might need to be set in the connection section.

If that is in fact the case it should be pretty easy to adapt this PR to support that as well. Do you have an environment provisioned or easily provisioned with the alternate DNS scenario where you could test this easily? I do not, so it might be a little bit more involved for us to validate exactly what config will work.

Would the setup happen to be or be analogous to the endpoint specific DNS setup when using privatelink and the vpc interface endpoints for private s3 access? If so i can inquire with our infra team on the feasibility on getting an account provisioned with a similar setup, so i can do some validation.

Ah, good point! I will verify the changes in my environment and update the PR tomorrow.

Yes, the Private link and custom VPC Endpoints is a viable test case for what I am trying to achieve!

Ah, good point! I will verify the changes in my environment and update the PR tomorrow.

Yes, the Private link and custom VPC Endpoints is a viable test case for what I am trying to achieve!

@tworcester Just following up to see if you had the opportunity to test this in your environment. No rush on our end, just wanted to make sure to reach out and see how it was going.

Apologies for the slow follow up, here is explicitly what I have deployed and working:

backups: |
  [default]
  region = ###ZARF_VAR_REGION###
  provider = AWS
  use_iam_profile = true
  host_base = s3.###ZARF_VAR_REGION###.###ZARF_VAR_S3_DOMAIN###
  host_bucket = %(bucket)s.s3.###ZARF_VAR_REGION###.###ZARF_VAR_S3_DOMAIN###
connection: |
  endpoint: https://s3.###ZARF_VAR_REGION###.###ZARF_VAR_S3_DOMAIN###
  provider: AWS
  region: ###ZARF_VAR_REGION###
  use_iam_profile: true
registry: |
  s3:
    bucket: ###ZARF_VAR_CLUSTER_NAME###-gitlab-registry
    region: ###ZARF_VAR_REGION###

Apologies for the slow follow up, here is explicitly what I have deployed and working:

backups: |
  [default]
  region = ###ZARF_VAR_REGION###
  provider = AWS
  use_iam_profile = true
  host_base = s3.###ZARF_VAR_REGION###.###ZARF_VAR_S3_DOMAIN###
  host_bucket = %(bucket)s.s3.###ZARF_VAR_REGION###.###ZARF_VAR_S3_DOMAIN###
connection: |
  endpoint: https://s3.###ZARF_VAR_REGION###.###ZARF_VAR_S3_DOMAIN###
  provider: AWS
  region: ###ZARF_VAR_REGION###
  use_iam_profile: true
registry: |
  s3:
    bucket: ###ZARF_VAR_CLUSTER_NAME###-gitlab-registry
    region: ###ZARF_VAR_REGION###

No worries whatsoever!

I'm assuming the ###ZARF_VAR_CLUSTER_NAME### is specific to your env and the existing way the buckets are named would still work.

Have you actually tested the registry functions? I would think an endpoint would be required there too and I thought the upstream chart had been updated so registry deployment failed when it couldnt make the s3 connection, but if you could verify you can push something to the container registry i would feel much more confident in this working config.

If so, we can look at updating the secret template to allow for this config setup.

Confirmed, the above works in three separate deployments right now and I have verified that docker/ folder in the registry bucket has layers stored in it!