Infra: Upload images to a public AWS ECR

[Haarolean]

We currently publish to github (and mirror to DockerHub), but I'll raise an issue and discuss the possibility of uploading the images to a public ECR as well.

Originally posted by @Haarolean in #23 (reply in thread)

TODO:

• public gallery (https://gallery.ecr.aws/s0v8j8q9/kafbat/kafka-ui)
• workflow

[yermulnik]

@Haarolean Have there been any updates on this recently? Thanks.

[yermulnik]

@Haarolean Just to reconfirm: this is the correct AWS Public ECR repo — https://gallery.ecr.aws/s0v8j8q9/kafbat/kafka-ui, right?

Can the s0v8j8q9 id be dropped from the path and the description added to the repo for visibility? Thank you.

[Haarolean]

@yermulnik yeah it's the right one. The id can't be dropped (unless you know something we don't :) ), the first deploy should be performed once #586 is merged and we'll update the description soon

[yermulnik]

Thank you.
We can expect version and latest tags to be added as well, right? 😺

[yermulnik]

The id can't be dropped (unless you know something we don't :) )

FWIW: https://aws.amazon.com/blogs/aws/amazon-ecr-public-a-new-public-container-registry/

ps: hope this helps as I haven't done this myself before 🤷🏻

[Haarolean]

Thank you. We can expect version and latest tags to be added as well, right? 😺

latest will be pushed there once 1.1 is released, will that work for you?

[yermulnik]

Thank you. We can expect version and latest tags to be added as well, right? 😺

latest will be pushed there once 1.1 is released, will that work for you?

Absolutely. Appreciate it 👍🏻

[yermulnik]

@Haarolean Given the number of tasks for 1.1 milestone, are there any approx timelines for that release? 🤔 Past due by 8 months 😢

FWIW: aws.amazon.com/blogs/aws/amazon-ecr-public-a-new-public-container-registry

Did I get it right, that you decided to not go with custom registry alias for a reason?

Also is the main tag re-added for every new commit'ish tag? (trying to figure out what we should rely upon unless releases are tagged correctly going forward).
Thanks.

[Haarolean]

are there any approx timelines for that release

Yeah, once #72 is merged in.

Did I get it right, that you decided to not go with custom registry alias for a reason?

Nope. I believe we didn't get the alias approved or something. I'll take a look into that.

Also is the main tag re-added for every new commit'ish tag

Yeah, that's right. It always points to the latest commit in main.
Besides the main and release tags, every other commit in main has its own tag. You can use one of these.

[yermulnik]

🥳 Yay, can see v1.1.0 (and latest) at https://gallery.ecr.aws/s0v8j8q9/kafbat/kafka-ui now. Thank a lot.

Any updates on custom registry alias in place of random id? Thanks.

[Haarolean]

@yermulnik dunno, they're quite stubborn:

Will try to investigate

[yermulnik]

That's odd 😕 Any chance you can create an AWS support request to see why that's rejected?

[yermulnik]

I also reached out to AWS Support on this case and here's what they provided:

Please allow me to mention the internal team has mentioned that the request was getting rejected because the email of the account holder does not match with the alias kafka requested as Kafka is a separate platform https://kafka.apache.org/ .

So as next step I would request you to please have a check on the email from which the request is being raised.

Maybe it is worth of requesting kafbat as an alias for this Public ECR or otherwise an alias that matches AWS account holder email (I'm supposing it's somehow related either to kafbat or to kafka-ua names 🤞🏻)? 🤔

[Haarolean]

I also reached out to AWS Support on this case and here's what they provided:

Please allow me to mention the internal team has mentioned that the request was getting rejected because the email of the account holder does not match with the alias kafka requested as Kafka is a separate platform https://kafka.apache.org/ .
So as next step I would request you to please have a check on the email from which the request is being raised.

Maybe it is worth of requesting kafbat as an alias for this Public ECR or otherwise an alias that matches AWS account holder email (I'm supposing it's somehow related either to kafbat or to kafka-ua names 🤞🏻)? 🤔

thanks for doing this! Indeed we had a same thought today, applied as kafbat-ui instead, let's see.

[yermulnik]

applied as kafbat-ui instead, let's see.

If you'll be pushing images to this registry for non-kafka-ui products developed by kafbat team, then the -ui suffix may look odd I guess.

[Haarolean]

applied as kafbat-ui instead, let's see.

If you'll be pushing images to this registry for non-kafka-ui products developed by kafbat team, then the -ui suffix may look odd I guess.

sorry you're right, we've applied kafbat instead, it's rather a typo in my message

[yermulnik]

JFYI:

Please allow me to mention that internal team has confirmed you can request for "kafbat" through console and it will be processed normally.

[Haarolean]

@yermulnik thank you!
it's done now: https://gallery.ecr.aws/kafbat/kafbat/kafka-ui
the name is kinda weird tho (kafbat/kafbat/kafka-ui)

[yermulnik]

Marvelous. Thanks! 👍🏻

[yermulnik]

the name is kinda weird tho (kafbat/kafbat/kafka-ui)

You probably need to drop now redundant s0v8j8q9/ from vars.ECR_REGISTRY value for docker tag 🤔

ps: I'm assuming that vars.ECR_REGISTRY is set to <aws_account_id>.dkr.ecr.us-east-1.amazonaws.com/s0v8j8q9 at the moment and hence (given the kafbat ECR alias is set now) it can be reduced to <aws_account_id>.dkr.ecr.us-east-1.amazonaws.com so that ${{ env.REGISTRY }}/${{ env.REPOSITORY }} resolves to <aws_account_id>.dkr.ecr.us-east-1.amazonaws.com/kafbat/kafka-ui 🤞🏻

[yermulnik]

Please let me know if that helps.

[yermulnik]

You also probably can simplify code in docker publish workflow by replacing define env vars step with the below code:

      - name: define env vars
        run: |
          if [ ${{ matrix.registry }} == 'ecr' ]; then
            echo "REGISTRY=${{ vars.ECR_REGISTRY }}" >> $GITHUB_ENV
            echo "REPOSITORY=${{ github.repository }}" >> $GITHUB_ENV
          else
            echo "REGISTRY=${{ matrix.registry }}" >> $GITHUB_ENV
            echo "REPOSITORY=${{ github.repository }}" >> $GITHUB_ENV
          fi

[Haarolean]

the name is kinda weird tho (kafbat/kafbat/kafka-ui)

You probably need to drop now redundant s0v8j8q9/ from vars.ECR_REGISTRY value for docker tag 🤔

ps: I'm assuming that vars.ECR_REGISTRY is set to <aws_account_id>.dkr.ecr.us-east-1.amazonaws.com/s0v8j8q9 at the moment and hence (given the kafbat ECR alias is set now) it can be reduced to <aws_account_id>.dkr.ecr.us-east-1.amazonaws.com so that ${{ env.REGISTRY }}/${{ env.REPOSITORY }} resolves to <aws_account_id>.dkr.ecr.us-east-1.amazonaws.com/kafbat/kafka-ui 🤞🏻

it was set to public.ecr.aws/s0v8j8q9. I just tried replacing it with public.ecr.aws with no luck:

unexpected status from POST request to https://public.ecr.aws/v2/kafbat/kafka-ui/blobs/uploads/: 404 Not Found
Error: Process completed with exit code 1.

[yermulnik]

it was set to public.ecr.aws/s0v8j8q9. I just tried replacing it with public.ecr.aws with no luck:

unexpected status from POST request to https://public.ecr.aws/v2/kafbat/kafka-ui/blobs/uploads/: 404 Not Found
Error: Process completed with exit code 1.

Ah, I see. Please try to set vars.ECR_REGISTRY back to public.ecr.aws/s0v8j8q9 and instead replace

          elif [ ${{ matrix.registry }} == 'ecr' ]; then
            echo "REGISTRY=${{ vars.ECR_REGISTRY }}" >> $GITHUB_ENV
            echo "REPOSITORY=${{ github.repository }}" >> $GITHUB_ENV

with

          elif [ ${{ matrix.registry }} == 'ecr' ]; then
            echo "REGISTRY=${{ vars.ECR_REGISTRY }}" >> $GITHUB_ENV
            echo "REPOSITORY=$(basename ${{ github.repository }})" >> $GITHUB_ENV

The gist of the change: REPOSITORY=${{ github.repository }} -> REPOSITORY=$(basename ${{ github.repository }})

[Haarolean]

@yermulnik wanna raise a PR for that?

[yermulnik]

wanna raise a PR for that?

Do you mean I should raise or you just want to allow me to contribute via PR?
If you think I'd better raise PR, I can do that. Otherwise I don't think it's worth of a fuss if you already made those changes and they worked as expected.

[Haarolean]

That's merely a contribution suggestion :) I haven't tried anything since your last comment yet

[yermulnik]

No problem. Let try this out: #794 (though it's still you to test the changes I guess 🤷🏻)

[yermulnik]

@Haarolean Any chance for #794 to get rolling? =)

[Haarolean]

@yermulnik
https://github.com/kafbat/kafka-ui/actions/runs/13051969523/job/36414297377

unexpected status from POST request to https://public.ecr.aws/v2/s0v8j8q9/kafka-ui/blobs/uploads/: 404 Not Found
Error: Process completed with exit code 1.

huh

[yermulnik]

Hmm, ah, I guess I figured it... I'm trying to come up with a correct path blindly as I obviously have no access to target infra of yours.

So I suppose that there's a kafbat/kafka-ui repo in your public ECR, but no kafka-ui repo. Please try and create kafka-ui repo:

[Haarolean]

@yermulnik indeed! This was the culprit.
https://gallery.ecr.aws/kafbat/kafka-ui
thanks for your help ❤️
P.S. I'll try to redeploy the release images to ecr, if I don't manage to do that, we'll keep the old (https://gallery.ecr.aws/kafbat/kafbat/kafka-ui) repo until the next release

[Haarolean]

pushed both past releases to https://gallery.ecr.aws/kafbat/kafka-ui

[yermulnik]

Yay 🥳 Thanks a bunch! Great collaboration experience ❤