[BUG] An error occurred (ValidationError) when calling the CreateChangeSet operation: Stack [microservices] already exists and cannot be created again with the changeSet [microserviceslisoguuxne].

[raffidahmad]

Describe the bug
A clear and concise description of what the bug is.

To Reproduce
Steps to reproduce the behavior:

1. How you installed ECS ComposeX: pip
2. How you are executing it ? : As a CLI tool

Docker compose file:

services:
  yarp:
    <<: *env
    container_name: yarp
    image: microservices/yarp:develop-${BITBUCKET_COMMIT}
    build:
      <<: *build-context
      dockerfile: ./APIGateway/Yarp/Dockerfile
    ports:
      - "55560:55560"
    deploy:
      <<: *deploy_config
      labels:
        ecs.task.family: proxy
      replicas: 1
  user-management:
    <<: *env
    container_name: user-management
    image: microservices/user-management:${TAG:-develop-${BITBUCKET_COMMIT}}
    depends_on:
      - yarp
    build:
      <<: *build-context
      dockerfile: ./Microservices/UserManagementService/Dockerfile
    ports:
      - "55567:55567" # for http #TODO: move to .env file + code + compose-x
#      - "55667:55667" # for grpc
    deploy:
      <<: *deploy_config
      labels:
        ecs.task.family: user-management
      replicas: 1

aws-compose-extension:

services:
  yarp:
    image: ${AWS_ACCOUNT_ID}.dkr.ecr.$AWS_REGION.amazonaws.com/microservices/yarp:develop-$BITBUCKET_COMMIT
    x-ecr:
      InterpolateWithDigest: true
    x-aws-role: # equivalent to x-iam https://docs.compose-x.io/syntax/ecs_plugin/support.html#x-aws-role
      <<: *ecs-task-role
  user-management:
    image: ${AWS_ACCOUNT_ID}.dkr.ecr.$AWS_REGION.amazonaws.com/microservices/user-management:${TAG:-develop-$BITBUCKET_COMMIT}
    x-ecr:
      InterpolateWithDigest: true
    x-aws-role:
      <<: *ecs-task-role

x-cluster:
  Lookup:
    ClusterName: compose-cluster
  Properties:
    CapacityProviders:
      - FARGATE
      - FARGATE_SPOT
    DefaultCapacityProviderStrategy:
      - CapacityProvider: FARGATE_SPOT
        Base: 1
        Weight: 1

x-elbv2:
  lb-microservices-develop:
    Lookup:
      loadbalancer:
        Tags:
          project: ntp
          env: develop
    Listeners:
      - Port: 55560
        Protocol: HTTPS
        Targets:
          - name: proxy:yarp
            access: /
        Certificates:
          - x-acm: cert-acm
      - Port: 55567
        Protocol: HTTPS
        Targets:
          - name: user-management:user-management:55567
            access: /
        Certificates:
          - x-acm: cert-acm
      - Port: 55667
        Protocol: HTTPS
        Targets:
          - name: user-management:user-management:55667
            access: /
        Certificates:
          - x-acm: cert-acm
#    TargetGroups:
#      yarp:
#        Port: 55560
#        Protocol: HTTP
#        HealthCheck: 55560:HTTP:2:2:30:5:/health:200
#        Services:
#          - Name: proxy:yarp
#            Port: 55560
#      user-management-default:
#        Port: 55567
#        Protocol: HTTP
#        HealthCheck: 55567:HTTP:2:2:30:5:/health:200
#        Services:
#          - Name: user-management:user-management
#            Port: 55567
#      user-management-grpc:
#        Port: 55667
#        Protocol: HTTP
#        HealthCheck: 55667:HTTP:2:2:30:5:/health:200,299
#        Services:
#          - Name: user-management:user-management
#            Port: 55667
    Services:
      proxy:yarp:
        port: 55560
        protocol: HTTP
        healthcheck: 55560:HTTP:2:2:30:5:/health:200
      user-management:user-management:
        port: 55567
        protocol: HTTP
        healthcheck: 55567:HTTP:2:2:30:5:/health:200
#      user-management:user-management:55667:
#        port: 55667
#        protocol: HTTP
#        healthcheck: 55667:HTTP:2:2:30:5:/health:200,299

Expected behavior
I first create my stack using only 1 service, it works
ecs-compose-x plan -f compose.yaml -f compose.ecs-extension.yaml -n microservices

Then i add another service, its listener (i cant add the :port that also fails for some reason)
and when i run the same plan command again
ecs-compose-x plan -f compose.yaml -f compose.ecs-extension.yaml -n microservices

everything works seemingly but when it's supposed to show the changeset it throws an error that the stack already exists. When it should just update the stack and add the new ones.
Or am i doing this wrong?

Logs
Error from compose-x after everything is done:
An error occurred (ValidationError) when calling the CreateChangeSet operation: Stack [microservices] already exists and cannot be created again with the changeSet [microserviceslisoguuxne].

Desktop (please complete the following information):

• OS: Linux/Windows
• Python version (3.12)
• Version: latest release

[JohnPreston]

Hello @raffidahmad
This seems to be due to the limitation that creating the changeset, which creates a stack (although does not execute, so does not create any resources), you can't plan over the same stack name. You would need to delete the root stack and nested stacks to be able to create a new "plan"

[raffidahmad]

Hello @raffidahmad This seems to be due to the limitation that creating the changeset, which creates a stack (although does not execute, so does not create any resources), you can't plan over the same stack name. You would need to delete the root stack and nested stacks to be able to create a new "plan"

Hi @JohnPreston , yes i was able to use "up" and continue on however i think it would be a good feature to have no? To be able to see the plan/changeset before doing the up operation.

[JohnPreston]

Yes, agreed, but then I need to handle the "already exists exception" and prompt the user whether they want to do a cleanup first then run the plan again. Does that make sense?

[raffidahmad]

I might be misunderstanding here but does it mean if i created a stack using up first, the next time i make a change and do "plan", i will first need to delete the stack i already made before?

Or do you mean a "plan" cannot happen over a "plan"

[JohnPreston]

I don't think so, no. But if the stack is not created but review in progress then I think that's where you get the exception. I am going to test all this and see what happens, I don't recall on top of my head TBH, I for pipelines purposes use create all the time

[raffidahmad]

Okay, I did not test out "create" for my use case but I'll try it out and see what happens.
I imagined "plan" more like git diff e.g. if im planning out my CI/CD process to include a step for cloudformation,

1. I make my compose files, do "plan" and see everything is "Add" in the result
2. Then i make some changes, do "plan" again, I expected it to show me the updates only instead of the exception

Were my expectations misaligned with what it is actually for?

[JohnPreston]

No, your exceptions are correct, I just never did implement the plan that retrieves the diff on an existing stack, I don't recall doing that anyways. But if a ChangeSet already exists, it has to be deleted or created again with a different name. So that could be the easy fix.