Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

github: rework Docker build, and publish new images #489

Draft
wants to merge 1 commit into
base: main
Choose a base branch
from
Draft

github: rework Docker build, and publish new images #489

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
87 changes: 87 additions & 0 deletions .github/workflows/docker.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,87 @@
# https://docs.github.com/en/free-pro-team@latest/actions/reference/workflow-syntax-for-github-actions
name: Build Docker Images
on:
# Manual trigger:
# https://docs.github.com/en/actions/reference/events-that-trigger-workflows#workflow_dispatch
workflow_dispatch:
# Scheduled trigger:
# https://docs.github.com/en/actions/reference/events-that-trigger-workflows#scheduled-events
schedule:
- cron: "0 1 * * mon" # every monday at 1am

env:
# We have to use 'glean' here, not 'Glean', because Docker does not allow
# uppercase letters in image names, so the image push will inevitably fail
BASE_IMAGE_NAME: ${{ github.repository_owner }}/glean
REGISTRY: ghcr.io

jobs:
build-docker:
name: Build and Push Docker Images
strategy:
matrix:
os: [ubuntu-24.04, ubuntu-24.04-arm]
runs-on: ${{ matrix.os }}
permissions:
contents: read
packages: write
steps:
- name: Checkout Repository
uses: actions/checkout@v4
with:
persist-credentials: false

- name: Log in to the Container registry
uses: docker/login-action@f4ef78c080cd8ba55a85445d5b36e214a81df20a # v2.1.0
with:
registry: ${{ env.REGISTRY }}
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}

- name: Derive image tags
run: |
DATE=$(date +%Y%m%d%H%M)
echo "Using date: ${DATE}"
echo "Using hash: ${GITHUB_SHA}"
echo "IMAGE_DATE_TAG=v${DATE}" >> $GITHUB_ENV
echo "IMAGE_SHA_TAG=${GITHUB_SHA}" >> $GITHUB_ENV
env:
GITHUB_SHA: ${{ github.sha }}

- name: Build and save Docker images
run: |
for x in server client demo; do
docker build -t $x:latest --target $x .
for v in latest ${IMAGE_DATE_TAG} ${IMAGE_SHA_TAG}; do
docker tag $x:latest ${REGISTRY}/${BASE_IMAGE_NAME}/$x:$v
done
done
env:
REGISTRY: ${{ env.REGISTRY }}
BASE_IMAGE_NAME: ${{ env.BASE_IMAGE_NAME }}
IMAGE_DATE_TAG: ${{ env.IMAGE_DATE_TAG }}
IMAGE_SHA_TAG: ${{ env.IMAGE_SHA_TAG }}

- name: Test Docker image
run: |
docker run --rm \
--entrypoint /usr/local/bin/glean \
demo:latest \
shell --db-root /glean/db --schema dir:/glean/schema \
':describe' \
':db react/0' \
':stat'

- name: Push image to the Container registry
run: |
set -x
for x in server client demo; do
for v in latest ${IMAGE_DATE_TAG} ${IMAGE_SHA_TAG}; do
echo docker push $x:latest ${REGISTRY}/${BASE_IMAGE_NAME}/$x:$v
done
done
env:
REGISTRY: ${{ env.REGISTRY }}
BASE_IMAGE_NAME: ${{ env.BASE_IMAGE_NAME }}
IMAGE_DATE_TAG: ${{ env.IMAGE_DATE_TAG }}
IMAGE_SHA_TAG: ${{ env.IMAGE_SHA_TAG }}
43 changes: 0 additions & 43 deletions .github/workflows/glean-docker.yml
View file
Open in desktop

This file was deleted.

212 changes: 131 additions & 81 deletions Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,96 +1,146 @@
FROM ghcr.io/facebookincubator/hsthrift/ci-base:latest as tools
# remove any old stuff
RUN rm -rf /usr/local/lib
RUN rm -rf /usr/local/include
RUN apt-get install -y ghc-8.10.2 cmake ninja-build libxxhash-dev wget unzip rsync libgmock-dev
RUN cabal update
RUN mkdir /glean-code
WORKDIR /glean-code
ADD https://api.github.com/repos/facebookincubator/hsthrift/compare/main...HEAD /dev/null
ADD ./glean /glean-code/glean
ADD ./thrift /glean-code/thrift
ADD ./cabal.project /glean-code/
ADD ./Makefile /glean-code/
ADD ./mk /glean-code/mk
ADD ./glean.cabal.in /glean-code/
ADD ./LICENSE /glean-code/
ADD ./Setup.hs /glean-code/
ADD ./install_deps.sh /glean-code/
RUN ./install_deps.sh
# Nuke build artifacts to save space
RUN rm -rf /tmp/fbcode_builder_getdeps-Z__wZGleanZGleanZhsthriftZbuildZfbcode_builder-root/
# MARK: build: deps+setup
FROM ubuntu:24.04 AS build
WORKDIR /build

ENV LD_LIBRARY_PATH=/root/.hsthrift/lib
ENV PKG_CONFIG_PATH=/root/.hsthrift/lib/pkgconfig
ENV PATH=$PATH:/root/.hsthrift/bin
# This is what is tested at Meta, so we use it here.
ENV GHC_VER=9.2.8
# See: https://github.com/haskell/cabal/issues/10046
ENV CABAL_VER=3.10
# Major available version of Clang.
ENV CLANG_VER=15
# Must be set for GHC, otherwise cabal will fail to build files with non-ASCII
# characters (like lens).
ENV LANG=en_US.UTF-8

RUN make
RUN cp $(cabal exec --project-file=cabal.project -- which glean) ~/.cabal/bin/
RUN cp $(cabal exec --project-file=cabal.project -- which glean-server) ~/.cabal/bin/
RUN cp $(cabal exec --project-file=cabal.project -- which glean-hyperlink) ~/.cabal/bin/
RUN glean --help
RUN wget https://github.com/facebook/flow/releases/download/v0.219.0/flow-linux64-v0.219.0.zip
RUN unzip flow-linux64-v0.219.0.zip
RUN mkdir -p /root/.hsthrift/bin && mv flow/flow /root/.hsthrift/bin/ && rm -rf flow-linux64-v0.219.0.zip flow/
WORKDIR /
RUN git clone https://github.com/facebook/react.git --depth 1 react-code
RUN cat /react-code/scripts/flow/config/flowconfig \
| grep -v REACT_RENDERER_FLOW_ \
| grep -v CI_MAX_WORKERS \
| grep -v FLOW_VERSION \
| sed '/^\[options\]/a exact_by_default=false' > /react-code/.flowconfig
# Install base dependencies.
RUN apt update \
&& apt install -y \
curl locales build-essential clang-$CLANG_VER git ninja-build make cmake libboost-all-dev \
libgmp-dev libaio-dev libbz2-dev libdouble-conversion-dev libdwarf-dev libgoogle-glog-dev libiberty-dev libjemalloc-dev \
libnuma-dev liblzma-dev liblz4-dev libsnappy-dev libsodium-dev libssl-dev libunwind-dev libzstd-dev libfast-float-dev \
pkg-config rsync libgmock-dev libpcre3-dev libtinfo-dev libxxhash-dev patchelf \
&& locale-gen en_US.UTF-8 && update-locale LANG=en_US.UTF-8

# Install GHC and Cabal via ghcup.
RUN curl --proto '=https' --tlsv1.2 -sSf "https://downloads.haskell.org/~ghcup/$(uname -m)-linux-ghcup" -o /tmp/ghcup \
&& chmod +x /tmp/ghcup \
&& /tmp/ghcup install cabal $CABAL_VER --set \
&& /tmp/ghcup install ghc $GHC_VER --set

FROM ubuntu:20.04 AS demo
# Use Clang, not gcc. Note that this MUST happen *AFTER* installing GHC above.
# Apparently `ghcup` presumably modifies the settings for GHC in some way, which
# inevitably causes a link failure due to incompatible -fPIC vs non-fPIC build
# settings...
RUN apt remove -y gcc g++ && apt autoremove -y \
&& update-alternatives --install /usr/bin/cc cc /usr/bin/clang-$CLANG_VER 10 \
&& update-alternatives --install /usr/bin/c++ c++ /usr/bin/clang++-$CLANG_VER 10 \
&& update-alternatives --install /usr/bin/gcc gcc /usr/bin/clang-$CLANG_VER 10 \
&& update-alternatives --install /usr/bin/g++ g++ /usr/bin/clang++-$CLANG_VER 10

LABEL org.opencontainers.image.source="https://github.com/facebookincubator/Glean"
# Run deps build (rocksdb, folly, hsthrift, etc). install_deps.sh needs some of
# the source files, so copy the minimum amount needed so that we can skip this
# step as much as possible (because these files are not changed) as frequently.)
COPY ./mk ./mk
COPY install_deps.sh Makefile cabal.project glean.cabal.in ./
RUN ./install_deps.sh --threads $(nproc)
RUN rm -rf /tmp/fbcode_builder_getdeps-Z__wZGleanZGleanZhsthriftZbuildZfbcode_builder-root/

ENV PATH=/glean-demo/bin:$PATH
ENV LD_LIBRARY_PATH=/usr/local/lib
# MARK: build: main build
COPY ./thrift/ ./thrift
COPY ./glean ./glean
RUN touch settings.mk \
&& echo "EXTRA_GHC_OPTS=-j$(nproc) +RTS -A128m -n2m -RTS" >> settings.mk \
&& echo "CABAL_CONFIG_FLAGS=-fclang -f-hack-tests -f-rust-tests -f-python-tests" >> settings.mk
RUN true && \
( export LD_LIBRARY_PATH=$HOME/.hsthrift/lib:$LD_LIBRARY_PATH \
; export PKG_CONFIG_PATH=$HOME/.hsthrift/lib/pkgconfig:$PKG_CONFIG_PATH \
; export PATH=$HOME/.ghcup/bin:$HOME/.hsthrift/bin:$PATH \
; cabal update && make -j$(nproc) && make bindist \
)
# Now patch the binaries to include /usr/local/lib in their rpath, as that's
# where the final libraries will go anyway.
RUN patchelf --add-rpath /usr/local/lib ./bindir/bin/*

RUN apt-get update && apt-get install -y \
libicu66 \
libboost-context1.71.0 \
libboost-filesystem1.71.0 \
libboost-program-options1.71.0 \
libboost-regex1.71.0 \
libunwind8 \
libgoogle-glog0v5 \
libssl1.1 \
libsodium23 \
libdouble-conversion3 \
libmysqlclient21 \
libevent-2.1-7 \
libsnappy-dev \
libxxhash0 \
libatomic1 \
&& apt-get clean && rm -rf /var/lib/apt/lists/*
# MARK: common: base image
FROM ubuntu:24.04 AS common
# This image only exists so that the client and server images can share it.
COPY --from=build /build/bindir/lib/* /usr/local/lib
RUN mkdir -p /glean/db /glean/schema \
&& apt update \
&& apt install -y \
libatomic1 \
libsnappy1v5 \
libpcre3 \
libunwind8 \
libgoogle-glog0v6t64 \
libicu74 \
libdouble-conversion3 \
libevent-2.1-7t64 \
libdwarf1 \
libsodium23 \
libaio1t64 \
libboost-context1.83.0 \
libboost-regex1.83.0 \
libboost-program-options1.83.0 \
libboost-system1.83.0 \
libboost-thread1.83.0 \
libboost-atomic1.83.0 \
libboost-filesystem1.83.0

WORKDIR /glean-demo
# MARK: server+client
FROM common AS client
COPY --from=build /build/bindir/bin/glean /usr/local/bin
CMD ["/usr/local/bin/glean", "--schema", "dir:/glean/schema"]

RUN mkdir /glean-demo/bin
FROM client AS server
COPY --from=build /build/bindir/bin/glean-server /usr/local/bin
CMD ["/usr/local/bin/glean-server", "--db-root", "/glean/db", "--schema", "dir:/glean/schema", "--port", "12345"]

COPY --from=tools /root/.hsthrift/lib /usr/local/lib
COPY --from=tools /root/.hsthrift/bin/flow /usr/local/bin
COPY --from=tools /root/.cabal/bin/glean /glean-demo/bin
COPY --from=tools /root/.cabal/bin/glean-server /glean-demo/bin
COPY --from=tools /root/.cabal/bin/glean-hyperlink /glean-demo/bin
COPY --from=tools /glean-code/glean/schema /glean-demo/schema
COPY --from=tools /react-code /glean-demo/code
ADD docker_entrypoint.sh docker_entrypoint.sh
FROM common AS tools
COPY --from=build /build/bindir/bin/glean /usr/local/bin
COPY --from=build /build/bindir/bin/gen-schema /usr/local/bin/glean-gen-schema

RUN mkdir -p db /tmp/flow-index-out
# MARK: demo setup
FROM common AS demo-build
RUN apt update && apt install -y unzip git
ADD https://github.com/facebook/flow/releases/download/v0.245.2/flow-linux64-v0.245.2.zip /tmp/flow-x86_64.zip
ADD https://github.com/facebook/flow/releases/download/v0.245.2/flow-linux-arm64-v0.245.2.zip /tmp/flow-aarch64.zip
RUN true && (cd /tmp; unzip flow-$(uname -m).zip)

RUN flow glean code --output-dir /tmp/flow-index-out --write-root "" && \
glean --db-root db --schema dir:schema/source create --repo react/0 && \
glean --db-root db --schema dir:schema/source write --repo react/0 /tmp/flow-index-out/* && \
glean --db-root db --schema dir:schema/source derive --repo react/0 flow.FileXRef flow.FileDeclaration && \
glean --db-root db --schema dir:schema/source finish --repo react/0 && \
rm -Rf /tmp/flow-index-out
RUN true \
&& git clone --depth 1 --branch v19.0.0 https://github.com/facebook/react.git /react-code \
&& rm -rf /react-code/.git
RUN cat /react-code/scripts/flow/config/flowconfig \
| grep -v REACT_RENDERER_FLOW_ \
| grep -v FLOW_VERSION \
| sed '/^\[options\]/a exact_by_default=false' > /react-code/.flowconfig

ENV REPO_NAME=react
FROM server AS demo
COPY --from=build /build/bindir/bin/glean-hyperlink /usr/local/bin
COPY --from=build /build/glean/schema/source /glean/schema/
COPY --from=demo-build /tmp/flow/flow /usr/local/bin
COPY --from=demo-build /react-code /glean/code/react

EXPOSE 8888
WORKDIR /glean
RUN mkdir -p /tmp/flow-index-out \
&& flow glean code/react --output-dir /tmp/flow-index-out --write-root "" \
&& glean --db-root db --schema dir:schema create --repo react/0 \
&& glean --db-root db --schema dir:schema write --repo react/0 /tmp/flow-index-out/* \
&& glean --db-root db --schema dir:schema derive --repo react/0 flow.FileXRef flow.FileDeclaration \
&& glean --db-root db --schema dir:schema finish --repo react/0 \
&& rm -rf /tmp/flow-index-out

ENTRYPOINT ["./docker_entrypoint.sh"]
# write out a simple inline script to start the demo app
RUN touch /usr/local/bin/glean-demo \
&& chmod a+x /usr/local/bin/glean-demo \
&& echo "#!/usr/bin/env bash" >> /usr/local/bin/glean-demo \
&& echo "set -ex" >> /usr/local/bin/glean-demo \
&& echo "/usr/local/bin/glean-server --db-root /glean/db --schema dir:/glean/schema --port 12345 &" >> /usr/local/bin/glean-demo \
&& echo "sleep 3" >> /usr/local/bin/glean-demo \
&& echo "exec /usr/local/bin/glean-hyperlink --service localhost:12345 --repo react --root /glean/code/react --http 8888" >> /usr/local/bin/glean-demo \
# Add a dummy favicon.ico so that the web server doesn't complain. \
&& touch /glean/code/react/favicon.ico

# docker run -ti -p8888:8888 ghcr.io/facebookincubator/glean/demo
EXPOSE 8888
EXPOSE 12345
ENTRYPOINT ["/usr/local/bin/glean-demo"]
11 changes: 10 additions & 1 deletion Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -132,9 +132,18 @@ cxx-libraries:
cxx-test-%: force
@$(MAKE) -f mk/cxx.mk --no-print-directory CXX_MODE=$(CXX_MODE) CXX_DIR=$(CXX_DIR) $@

GLEAN_CABAL_BINS=glean glean-server glean-hyperlink gen-schema
GLEAN_NATIVE_LIBS=librocksdb.so.8 libfolly.so.0.58.0-dev libfmt.so.11
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this looks fragile, can we make it independent of the version numbers?

Copy link
Contributor Author

@thoughtpolice thoughtpolice Jan 24, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe we can just do librocksdb.so.* or something like that and let bash do the rest of the magic for us

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I looked into this but it's not quite that simple, because just copying everything with an asterisk copies way too much and bloats up the container again. Instead, I think I'll just make sure that the built docker container has test queries run against it, which should ensure that the proper shared libraries are available (otherwise it wouldn't launch), so at least a break will be obvious and reported.

.PHONY: glean
glean:: glean.cabal cxx-libraries
$(CABAL) build glean glean-server glean-hyperlink
$(CABAL) build $(GLEAN_CABAL_BINS)

CABAL_BUILD_PREFIX := dist-newstyle/build/$(shell uname -m)-linux/ghc-$(shell ghc --numeric-version)/glean-0.1.0.0/x
.PHONY: bindist
bindist: glean
for b in $(GLEAN_CABAL_BINS); do install -D $(CABAL_BUILD_PREFIX)/$$b/build/$$b/$$b bindir/bin/$$b; done
for l in $(GLEAN_NATIVE_LIBS); do install -D $(HOME)/.hsthrift/lib/$$l bindir/lib/$$l; done
strip bindir/bin/* bindir/lib/*

.PHONY: gen-bytecode
gen-bytecode: $(BYTECODE_GEN)
Expand Down
2 changes: 1 addition & 1 deletion README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
[![CI](https://github.com/facebookincubator/Glean/actions/workflows/ci.yml/badge.svg)](https://github.com/facebookincubator/Glean/actions/workflows/ci.yml)
[![Glean demo Docker image](https://github.com/facebookincubator/Glean/actions/workflows/glean-docker.yml/badge.svg)](https://github.com/facebookincubator/Glean/actions/workflows/glean-docker.yml)
[![Glean demo Docker image](https://github.com/facebookincubator/Glean/actions/workflows/docker.yml/badge.svg)](https://github.com/facebookincubator/Glean/actions/workflows/docker.yml)

# Glean

Expand Down
Loading
Loading