Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

gvisor and network=host #11388

Open
kosmits-ai opened this issue Jan 24, 2025 · 5 comments
Open

gvisor and network=host #11388

kosmits-ai opened this issue Jan 24, 2025 · 5 comments
Labels
type: bug Something isn't working

Comments

@kosmits-ai
Copy link

kosmits-ai commented Jan 24, 2025

Description

Hello,

I'm using gVisor with Docker Compose and have encountered a network-related issue. From what I understand, when you create a user-defined bridge network, Docker sets up a DNS system for the containers. However, containers running with runsc can't access this DNS system due to the sandbox, and as a workaround, I have been using static IPs for communication between services.

The problem arises when I set the network of runsc to host, as my services can then reach external resources like github.com. Without setting network=host, my services can communicate with each other, but they cannot reach external resources like GitHub. Then i get fatal: unable to access 'https://github.com/<github_user>/<repo_name>/': Could not resolve host: github.com.This happens for every repo i try to access.

My question is: if I use network=host, do I lose all the security advantages that gVisor provides? Is there a safer workaround for allowing my containers to access external resources while maintaining gVisor's security benefits?

Thanks in advance for your help!

Steps to reproduce

No response

runsc version

runsc version release-20250113.0
spec: 1.1.0-rc.1

docker version (if using docker)

Docker version 27.5.0, build a187fa5

uname

No response

kubectl (if using Kubernetes)

repo state (if built from source)

No response

runsc debug logs (if available)

@kosmits-ai kosmits-ai added the type: bug Something isn't working label Jan 24, 2025
@milantracy
Copy link
Contributor

  • if I use network=host, do I lose all the security advantages that gVisor provides?

the short answer is that it will be less secured on networking

please see https://gvisor.dev/docs/user_guide/networking/#network-passthrough, using network=host trades the security and isolation of netstack for the performance of native Linux networking.

  • Is there a safer workaround for allowing my containers to access external resources while maintaining gVisor's security benefits?

I can't reproduce the issue by

$ docker run --runtime=runsc-host --rm -it test git clone --depth 1 https://github.com/google/gvisor.git
Cloning into 'gvisor'...
remote: Enumerating objects: 4199, done.
remote: Counting objects: 100% (4199/4199), done.
remote: Compressing objects: 100% (3829/3829), done.
remote: Total 4199 (delta 958), reused 1739 (delta 289), pack-reused 0 (from 0)
Receiving objects: 100% (4199/4199), 15.96 MiB | 8.50 MiB/s, done.
Resolving deltas: 100% (958/958), done.


# runsc-host in /etc/docker/daemon.json

        "runsc-host": {
            "path": "/tmp/runsc/runsc",
            "runtimeArgs": [
                "--directfs=false",
		"--network=host"
            ]
        },

I would like to see more from your runsc config and docker config

@kosmits-ai
Copy link
Author

This is my docker compose.

services:
  mongo-service:
    image: kosmits/thecrimepoirot:mongo-service-1.0.1
    ports:
      - "5000:5000"
    env_file:
      - .env
    runtime: runc
    networks:
      crimepoirot_network:
        ipv4_address: ${MONGO_SERVICE_IP}
    dns:
      - 8.8.8.8  # Google DNS
      - 8.8.4.4  # Google DNS

  gitleaks:
    image: kosmits/thecrimepoirot:gitleaks-1.0.1
    ports:
      - "5001:5001"
    env_file:
      - .env
    networks:
      crimepoirot_network:
        ipv4_address: ${GITLEAKS_IP}
    depends_on:
      - mongo-service
    runtime: runsc
    volumes:
      - /home/vboxuser/Documents/RepoForTest:/app/RepoForTest
    dns:
      - 8.8.8.8  # Google DNS
      - 8.8.4.4  # Google DNS

  guarddog:
    image: kosmits/thecrimepoirot:guarddog-1.0.1
    ports:
      - "5002:5002"
    env_file:
      - .env
    networks:
      crimepoirot_network:
        ipv4_address: ${GUARDDOG_IP}
    depends_on:
      - mongo-service
    runtime: runsc
    volumes:
      - /home/vboxuser/Documents/RepoForTest:/app/RepoForTest
    dns:
      - 8.8.8.8  # Google DNS
      - 8.8.4.4  # Google DNS

  safety:
    image: kosmits/thecrimepoirot:safety-1.0.1
    ports:
      - "5003:5003"
    env_file:
      - .env
    networks:
      crimepoirot_network:
        ipv4_address: ${SAFETY_IP}
    depends_on:
      - mongo-service
    runtime: runsc
    volumes:
      - /home/vboxuser/Documents/RepoForTest:/app/RepoForTest
    dns:
      - 8.8.8.8  # Google DNS
      - 8.8.4.4  # Google DNS

  bearer:
    image: kosmits/thecrimepoirot:bearer-1.0.1
    ports:
      - "5004:5004"
    env_file:
      - .env
    networks:
      crimepoirot_network:
        ipv4_address: ${BEARER_IP}
    runtime: runsc
    depends_on:
      - mongo-service
    volumes:
      - /home/vboxuser/Documents/RepoForTest:/app/RepoForTest
    dns:
      - 8.8.8.8  # Google DNS
      - 8.8.4.4  # Google DNS

  calculate_percentile:
    image: kosmits/thecrimepoirot:calculate_percentile-1.0.1
    ports:
      - "5005:5005"
    env_file:
      - .env
    volumes:
      - ./report.csv:/app/report.csv
      - /home/vboxuser/Documents/RepoForTest:/app/RepoForTest
    runtime: runc
    networks:
      crimepoirot_network:
        ipv4_address: ${PERCENTILE_SERVICE_IP}
    depends_on:
      - mongo-service
    dns:
      - 8.8.8.8  # Google DNS
      - 8.8.4.4  # Google DNS

  api_gateway:
    image: kosmits/thecrimepoirot:api_gateway-1.0.1
    ports:
      - "5007:5007"
    env_file:
      - .env
    networks:
      crimepoirot_network:
        ipv4_address: ${API_GATEWAY_IP}
    runtime: runc
    depends_on:
      - gitleaks
      - guarddog
      - safety
      - bearer
      - calculate_percentile
    volumes:
      - /home/vboxuser/Documents/RepoForTest:/app/RepoForTest
    dns:
      - 8.8.8.8  # Google DNS
      - 8.8.4.4  # Google DNS

  frontend:
    image: kosmits/thecrimepoirot:frontend-1.0.2
    ports:
      - "8501:8501"
    env_file:
      - .env
    volumes:
      - ./images:/app/images
      - /home/vboxuser/Documents/RepoForTest:/app/RepoForTest
    runtime: runc
    networks:
      crimepoirot_network:
        ipv4_address: ${FRONTEND_IP}
    depends_on:
      - api_gateway
    dns:
      - 8.8.8.8  # Google DNS
      - 8.8.4.4  # Google DNS

  update_db:
    image: kosmits/thecrimepoirot:update_db-1.0.1
    ports:
      - "5008:5008"
    env_file:
      - .env
    runtime: runc
    networks:
      crimepoirot_network:
        ipv4_address: ${UPDATE_DB_IP}
    depends_on:
      - mongo-service
      - gitleaks
      - guarddog
      - safety
      - bearer
      - api_gateway
    volumes:
      - ./report.csv:/app/report.csv
      - /home/vboxuser/Documents/RepoForTest:/app/RepoForTest
    dns:
      - 8.8.8.8  # Google DNS
      - 8.8.4.4  # Google DNS

networks:
  crimepoirot_network:
    external: true
    driver: bridge
    ipam:
      config:
        - subnet: ${CRIMEPOIROT_SUBNET}

When i use in docker config /etc/docker/daemon.json:

{
    "runtimes": {
        "runsc": {
            "path": "/usr/local/bin/runsc",
            "runtimeArgs": [
                "--network=host"
            ]
       }
    }
}

services communicate each other and can access github.com which is external.

When i use in docker config /etc/docker/daemon.json :

{
    "runtimes": {
        "runsc": {
            "path": "/usr/local/bin/runsc"
       }
    }
}

services communicate with each other but cant resolve github.com.
[According to gvisor documentation] (https://gvisor.dev/docs/user_guide/faq/#docker-bridge) in the question: My container cannot resolve another container’s name when using Docker user defined bridge, I used static IPs for the communication of containers.
So I need to find a way for gvisor to run without network=host for keeping my security in high levels but in the same time the services running in runsc must access github.com

@kosmits-ai
Copy link
Author

Also ,defining dns in compose doesnt seem to work for runsc containers.As far as I have understand when you create a define bridge network in docker it creates a DNS which is not accessible from runsc environment.So I thought that i could add DNS 8.8.8.8 in order to give DNS to this services but it did not work

@kosmits-ai
Copy link
Author

Any comments/ suggestions?

@johnwmail
Copy link

how about bind mount /etc/resolv.conf in container?
volumes:
- /etc/resolv.conf:/etc/resolv.conf:ro

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type: bug Something isn't working
Projects
None yet
Development

No branches or pull requests

3 participants