First effort with docker #4

davegill · 2019-02-27T16:23:02Z

No description provided.

…irectory

This brings in the new GNU compiler. If the standard GNU compiler on Linux is too old to build WRF, this is what is required. The side effect is that the "yum install netcdf" commands are now generating inconsistent Fortran *.mod files. The "yum install" command is defaulting to an assumed GNU version (4.8.5, OLD). With some source changes in WRF, there is no longer a need for the Fortran MODULE "use netcdf" statment, we can get away with "include netcdf.inc".

Dockerfile has GNU 8 (fixes RRTMG compile fails with -d) Dockerfile builds netcdf, hdf5, openmpi Dockerfile fixes env and various LD_LIB and PATHs Dockerfile and .travis.yml have option for regtest arg Includes small data option (No DA, no HWRF, No KPP) Includes better script with nml test number on pass Includes better script with defined $d on run failures Includes better script with stacksize unlimited Takes a while to build the image now, 40+ minutes, likely due to the building of those !@#$%^&* libraries from source Changes to be committed: modified: .travis.yml modified: Dockerfile modified: Dockerfile-template

… make install Stick files in /wrf/wrfoutput. The output was enormous, and travis complained. If the builds work, who looks at that stuff anyways, right? Changes to be committed: modified: Dockerfile modified: Dockerfile-template

Changes to be committed: modified: Dockerfile modified: Dockerfile-template

…top the build This is due to the travis requirement that the std out and err output from the configure and make of the external libs (netcdf, hdf5, openmpi) was too large. So, you stick the std out and err in a file to be a nice guy. Is travis happy? No. Now it sees no output for a long time (since the three libs take a while) and says "Hey, I'll bet this process is just hung". Travis wants to see output every 10 minutes. Instead of modifying the .travis.yml file, I tried to manufacture output after each of the configures (4 of them) and after each of the makes (4 of them). Fingers crossed. Sheesh. Changes to be committed: modified: Dockerfile modified: Dockerfile-template

Trying to dump piles of print for openmpi make Changes to be committed: modified: Dockerfile modified: Dockerfile-template

module_ra_rrtmg_swf.f90:3419:0: use rrsw_kg21_f, only : absa, ka, absb, kb, forref, selfref, & internal compiler error: in gfc_trans_use_stmts, at fortran/trans-decl.c:5111 Please submit a full bug report, Removing -d, seems to mostly be OK then with RRTMG fast sw then. But on Darwin and cheyenne, the GNU 8 fixed this. Also, adding -j 3 for the parallel build. Changes to be committed: modified: .travis.yml

Modified the associated script.csh to always bypass the RRTMG FAST build Changes to be committed: modified: .travis.yml

…in container

1. Has build WRF and repeat if necessary script.csh

Changes to be committed: modified: .travis.yml

…s work?

…line syntax, maybe?

…tuff

1. cenv gets {}, not required, but conventional for script 2. zap entire "addons" with the "apt" "ubuntu" packages" and "gcc" stuff 3. remove language=fortran, never used sine we are doing containers 4. make branches of interest: master, release-v*, and develop

…kerhub version

…the team

… builds complete

… get file names to search for

…internal failures intermittently

…nce not using -d)

davegill added 30 commits February 27, 2019 11:21

First effort with docker

9ccfbb2

Explicitly look inside container for executables

d602f30

Find out list of TRAVIS variables

4efc962

Try to edit Dockerfile to get correct WRF version

4d3d846

yum install git in Dockerfile

ce48326

Now add a run phase for WRF

c0468c0

Absolute path required with docker -w

bea504d

Remove the "-w" option from docker, does not work to change working d…

43d5a1f

…irectory

If BUILD fails: 1) output compile log, 2) do not try to run

3f24a80

gcc-6 -> gcc-8

35032ef

cat the wrf build log, not just an ls, we want to see what failed

0d6c3d0

docker ENV includes LD_LIBRARY_PATH and PATH with devtoolset

3da4e71

Write lib std out and err to /wrf/libs, since that dir exists (ahem)

165d834

Changes to be committed: modified: Dockerfile modified: Dockerfile-template

Still wrestling with 10 minute rule

efa6de8

Trying to dump piles of print for openmpi make Changes to be committed: modified: Dockerfile modified: Dockerfile-template

Removing -d made compile take too long, put -d back in

9d62b19

Modified the associated script.csh to always bypass the RRTMG FAST build Changes to be committed: modified: .travis.yml

For OpenMPI build, try awk to print every 1000th line - woot

35011e6

Just print out tons of openmpi make stuff, easier

93ea25c

Add in some debug prints to .travis.yml, what is $OK

e0af586

Lots of .travis.yml prints concerning return code for file existence …

8733b49

…in container

Fix sh if and or syntax oops in .travis.yml

9bcf542

Yet more syntax probs, freaking learn sh, dude

cb2921d

Uses second of the two-part Docker build, should be FAST

cbc6116

Prints in .travis.yml to see what is happening with mpirun command

04e46bb

prints in .travis.yml, using too many "slots" with MPI

8f45639

Use latest Dockerfile (not first try, oops)

d3ee983

davegill added 20 commits April 20, 2019 15:00

Use thirdtry Dockerfile

e784e93

1. Has build WRF and repeat if necessary script.csh

Supply NP=3, for env setting for MPI - how many MPI ranks

42dc9a8

Changes to be committed: modified: .travis.yml

This is the 4th try for using the faster Dockerfile

9e5e5b9

Remove tests to see how many MPI ranks were available

08416bc

.travis.yml: use matrix for three em_real jobs at same time

4efda61

.travis.yml: maybe syntax error? adding space after : in matrix lines

a28cf89

.travis.yml: back off to a single test, still a single line, does thi…

26c930a

…s work?

Hmmm, well back to a previous .travis.yml, does this STILL work?

bc39f5d

.travis.yml: OK that was fine, now swap : for = in matrix for single …

e54a2e1

…line syntax, maybe?

.travis.yml: make all my variables ENV

463edfd

.travis.yml: make env: then matrix:, get rid of all os linux trusty s…

4ae5675

…tuff

.travis.yml and Dockerfile-template: 3 matrix tests, use fifthtry doc…

194d0f3

…kerhub version

.travis.yml: add NMM tests to ARW, 5 total builds

992efcb

.travis.yml: all 24 builds, with one test each - chem may bring down …

b19bbf7

…the team

.travis.yml: Zap both chem tests, take too long for travis

0d094f3

.travis.yml: add travis_wait in front of WRF build, see if that helps…

9aeaab9

… builds complete

.travis.yml: add back chem with travis wait, include ls in WRF dir to…

103814a

… get file names to search for

.travis.yml - zap chem for good; try to build without -d, rrtmg sw K …

e4c79af

…internal failures intermittently

.travis.yml: bump up the travis wait to 25 min for the build step (si…

cfe9a3c

…nce not using -d)

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

First effort with docker #4

First effort with docker #4

Uh oh!

davegill commented Feb 27, 2019

Uh oh!

Uh oh!

First effort with docker #4

Are you sure you want to change the base?

First effort with docker #4

Uh oh!

Conversation

davegill commented Feb 27, 2019

Uh oh!

Uh oh!