Skip to content

Commit 6ee559d

Browse files
committed
Automate build and publish
1 parent 65d8a3b commit 6ee559d

20 files changed

+330
-148
lines changed

.gitignore

+1
Original file line numberDiff line numberDiff line change
@@ -77,3 +77,4 @@ larsoft-cfg
7777
configure
7878

7979
*.gz
80+
styles

apps/README.org

+2
Original file line numberDiff line numberDiff line change
@@ -134,3 +134,5 @@ top-level execution. Provided here are ~ConfigDumper~ and ~NodeDumper~
134134
which are more examples than useful. See ~TbbFlow~ from the ~tbb~ sub
135135
package and ~Pgrapher~ from the ~pgraph~ package for the two most used
136136
apps.
137+
138+

aux/README.org

-2
Original file line numberDiff line numberDiff line change
@@ -24,5 +24,3 @@ Some categories of code appropriate for ~aux~:
2424
- [[file:docs/ClusterArrays.org][array representation of cluster graphs]]
2525
- [[file:docs/cluster-shadow.org][the "cluster shadow" concept]]
2626

27-
28-

cfg/README.org

-7
Original file line numberDiff line numberDiff line change
@@ -73,13 +73,6 @@ layers* form introduced above. In some cases, legacy detector
7373
configuration may used in an adapter that provides *structured layers*
7474
forms.
7575

76-
** Obsolete
77-
78-
Configuration can become obsolete when the C++ changes or when new
79-
paradigms of configuration organization are adopted. That
80-
configuration may be moved to [[file:obsolete/]] for some time before being
81-
dropped from the tip of the master branch.
82-
8376
* Tests
8477

8578
As with other WCT subpackages, this provides a ~/test/~ sub-directory with various tests that used through the [[file:../test/README.org][WCT testing framework]]. In particular the ~cfg~ sub-package has many tests written in Jsonnet and the [[file:../test/docs/jsonnet.org][Jsonnet test support]] is particularly relevant.

docs/README.org

+167-35
Original file line numberDiff line numberDiff line change
@@ -4,62 +4,194 @@
44

55
* Overview
66

7-
The ~docs/~ sub package provides support for various forms of
8-
documentation provided by the ~wire-cell-toolkit~ repository.
9-
It integrates with the ~wire-cell-docs~ repository.
7+
The ~docs/~ sub package provides support for various forms of documentation provided by the ~wire-cell-toolkit~ repository.
108

11-
* Source
9+
* Documentation source
1210

13-
Most of the documentation source file is written in [[https://orgmode.org/][org markup]]. Each
14-
file is expected to produce an associated export file (eg ~foo.org~
15-
produces ~foo.html~). That is, authors should avoid use of ~#+include:~
16-
directives to aggregate files. In some special cases, documentation
17-
source files may be aggregated for the purpose of producing a
18-
monolithic export. Particular care is needed so that links resolve
19-
properly.
11+
The documentation text source is written in plain text following [[https://orgmode.org/][org markup]].
2012

21-
* README
13+
** README
2214

23-
At [[file:../README.org][top level]] and in every sub package in the WCT repo there is a
24-
~README.org~ file. These files provide entry points into the
25-
documentation for their context (top or sub package). They should be
26-
brief and delegate to other documents under the sibling directory
27-
~docs/~.
15+
At [[file:../README.org][top level]] and in every sub package in the WCT repo there is a ~README.org~ file. These files should provide a brief explanation of their context and then delegate details to other documents under the sibling directory ~docs/~.
2816

29-
Every ~README.org~ file must use the ~setup-readme.org~ setup file. For
30-
those in a sub-package they must start with lines like:
17+
Every ~README.org~ file should start with lines like:
3118

3219
#+begin_example
3320
,#+title: Wire-Cell Toolkit <Package>
3421
,#+include: ../docs/include-readme.org
3522
#+end_example
3623

37-
* Topic files
24+
This will allow the file to be consistently integrated with publishing (see below).
3825

39-
The WCT /topics/ documents provide the bulk of the WCT in-source
40-
documentation. They are placed under ~<pkg>/docs/~ and written to
41-
assume in-place export from that location (eg, for linking to or
42-
including source from other files). A topic document must use the
43-
~setup-topic.org~ setup file by using initial lines such as:
26+
** Topic files
27+
28+
A /topic/ file holds some detailed documentation on some aspect. It should be placed under ~<pkg>/docs/<topic>.org~.
29+
30+
Every /topic/ file should start with lines like:
4431

4532
#+begin_example
4633
,#+title: Wire-Cell Toolkit Topic XYZ
4734
,#+include: ../../docs/include-topic.org
4835
#+end_example
4936

37+
* Building
38+
39+
Each doc source file may be /exported/ to HTML or PDF manually (eg by ~emacs~ or less nicely by ~pandoc~). The build system performs this export automatically if Emacs is available. See also [[Publishing][Publishing]] below.
40+
41+
** Automatic
42+
43+
By default, the docs are not built. Their build is initiated with the command line option ~--docs=<...>~. It takes a list of transformations. Currently all transformations can be exercised with:
44+
45+
#+begin_example
46+
waf --docs=org2html,org-pdf
47+
#+end_example
48+
49+
This will build all documents.
50+
51+
** Individual
52+
53+
Individual documents can be built via the usual Waf target mechanism:
54+
55+
#+begin_example
56+
waf --docs=org2html list
57+
waf --docs=org2html --target=docs-README-html
58+
waf --docs=org2pdf --target=docs-README-pdf
59+
evince build/docs/README.pdf
60+
#+end_example
61+
62+
** Manual
63+
64+
Of course it is possible to manually transform the org documents to HTML or PDF. Care must be taken to not leave the exported products in the source tree. Do not ever commit generated documentation to the repo.
65+
66+
*** Using Emacs
67+
68+
Visit the org file and type ~C-c C-e h h~ to export it to HTML. Alternatively, ~C-c C-e l p~ will export to PDF.
69+
70+
#+begin_note
71+
Results may differ from officially published version. In particular, code syntax highlighting is sensitive Emacs customization.
72+
#+end_note
73+
74+
*** Using pandoc
75+
76+
If you are afraid of Emacs you can use ~pandoc~ to generate previews. For example:
77+
78+
#+begin_example
79+
cd docs
80+
pandoc -o README.html README.org
81+
pandoc -o README.pdf README.org
82+
#+end_example
83+
84+
#+begin_note
85+
The results with pandoc will be rather plain and ugly compared to what is produced by Emacs.
86+
#+end_note
87+
88+
89+
* Authoring
90+
91+
There is a huge body of guidance on authoring documents in org markup. Here, only a few special things relevant to WCT documentation are given. To get started:
92+
93+
#+begin_example
94+
git clone [email protected]:wirecell/wire-cell-toolkit.git /path/to/wct
95+
cd /path/to/wct
96+
$EDITOR <pkg>/docs/somefile.org
97+
#+end_example
98+
99+
Any text editor is fine to use though of course Emacs provides the most support.
100+
101+
** File links
102+
103+
In addition to linking to web URLs, relative links to other files are allowed. The form that should be used is:
104+
105+
#+begin_example
106+
See [[file:other.org][this other file]] for more info.
107+
#+end_example
108+
109+
Publishing will include the targets of these links.
110+
111+
** Images
112+
113+
Documents may include images. They are typically placed in the ~docs/~ directory alongside the org source.
114+
115+
#+begin_example
116+
Enjoy this figure:
117+
118+
[[file:/wire-cell-bee-logo.png]]
119+
#+end_example
120+
121+
[[file:wire-cell-bee-logo.png]]
122+
123+
Graphics files can pose a problem for allowing the org source to export to both HTML and PDF (by way of LaTeX). Use of PNG and JPEG is compatible with both but for some content leads to poor resolution or large files. Use of PDF images is best for PDF export and for line-art but PDF will not be inlined in HTML export and will instead be presented as a link. In principle, SVG can be used for both and has good resolution/size features but use in PDF/LaTeX export requires some special attention.
124+
125+
** File inclusion
126+
127+
Avoid including other org files as this causes ambiguity in relative links and goes against the "one org file per document" pattern that the build assumes.
128+
129+
* Previewing
130+
131+
While editing it can be useful to see a rendered version. One may simply build a document as described in [[Building][Building]]. This can be coupled with a "live view" previewing.
132+
133+
** PDF
134+
135+
The ~evince~ or ~atril~ PDF viewers provide "live view", reloading when the PDF updates.
136+
137+
** HTML
138+
139+
A "live view" of the HTML file can be useful as one edits. This can approximate "instant" publishing. It may be used like:
140+
141+
#+begin_example
142+
cd /path/to/wct
143+
ln -s docs/styles .
144+
livereload -d -p 5959 -t .
145+
#+end_example
146+
The reason for the symlink is to make the source area look like the published web area.
147+
148+
#+begin_tip
149+
The command may be installed with ~pip install livereload~. It may be convenient to install this in the same Python environment as ~wire-cell-python~.
150+
#+end_tip
151+
152+
#+begin_note
153+
File tree indices (~index.html~) are not automatically generated by ~livereload~ so you will must type in the full path to an HTML file in the URL. For example, the top level README would be at the URL http://127.0.0.1:5959/README.html.
154+
#+end_note
50155

51-
* HTML
156+
* Publishing
52157

53-
Each WCT documentation file may be exported in-place to an HTML file.
54-
Links should be relative. However, CSS and javascript files are found
55-
using an absolute URL rooted at ~/~. When the exported HTML is
56-
installed to the ~wirecell.github.io~ package and served from that
57-
domain or https://wirecell.bnl.gov/ this path will be assured. When
58-
developing the documentation it is useful to run a local server such
59-
as with:
158+
We publish the WCT source docs by "installing" the exported HTML files into the ~git~ repo that holds the WCT web site.
60159

61160
#+begin_example
62-
$ pip install livereload
63-
$ livereload -p 5959 -t /path/to/wire-cell-toolkit
161+
git clone [email protected]:WireCell/wirecell.github.io /path/to/web
162+
git clone [email protected]:wirecell/wire-cell-toolkit.git /path/to/wct
163+
164+
cd /path/to/wct
165+
waf configure [...]
166+
waf install --docs=org2html --docs-prefix=/path/to/web
167+
168+
cd /path/to/web
169+
git commit
170+
git push
64171
#+end_example
65172

173+
#+begin_tip
174+
By default, the ~waf install --docs=org2html~ command will launch a new instance of Emacs for each org file. This can be slow. An optional optimization is supported that will speed up the build by about 4x. To use it, first start ~emacs~ and there issue ~M-x (server-start)~. Then build with: ~waf install --docs=org2html --emacs-daemon="server"~.
175+
#+end_tip
176+
177+
* Other markup
178+
179+
Org markup syntax is very simple and plain text. You may use any editor to author it. Above gives various ways to convert to HTML or PDF without using or knowing Emacs.
180+
181+
In principle, other markup formats could be supported, but this is discouraged. Supporting a new markup format requires a developer to "champion" the formation. Requirement on the developer include:
182+
183+
- Demonstrate that they will actually provide and maintain documentation.
184+
185+
- Explain why they are incapable of using org.
186+
187+
- Develop a waf tool to integrate the formst that operates symmetric with the [[file:../waft/org.py][org waf tool]].
188+
189+
- Provide HTML CSS/JS styling.
190+
191+
* Other documentation
192+
193+
WCT has additional documentation beyond what is housed in the source repo. See entry point at:
194+
195+
- https://wirecell.bnl.gov/
196+
197+
- https://wirecell.github.io/ (mirror)

docs/include-readme.org

+1-1
Original file line numberDiff line numberDiff line change
@@ -1,2 +1,2 @@
11
#+SETUPFILE: ../docs/setup-readme.org
2-
#+subtitle: [[file:../README.org][(top)]]
2+
#+html: <center><a href="../README.html">(top)</a></center>

docs/include-topic.org

+2-1
Original file line numberDiff line numberDiff line change
@@ -1,2 +1,3 @@
11
#+SETUPFILE: ../../docs/setup-topic.org
2-
#+subtitle: [[file:../../README.org][(top)]] [[file:../README.org][(pkg)]]
2+
#+html: <center><a href="../README.html">(top)</a> <a href="README.html">(pkg)</a></center>
3+

docs/styles/setup-bigblow.org

+10-10
Original file line numberDiff line numberDiff line change
@@ -1,15 +1,15 @@
11
# -*- mode: org; -*-
22

33
#+OPTIONS: html-style:nil
4-
#+HTML_HEAD: <link rel="stylesheet" type="text/css" href="docs/styles/bigblow/css/htmlize.css"/>
5-
#+HTML_HEAD: <link rel="stylesheet" type="text/css" href="docs/styles/bigblow/css/bigblow.css"/>
6-
#+HTML_HEAD: <link rel="stylesheet" type="text/css" href="docs/styles/bigblow/css/hideshow.css"/>
4+
#+HTML_HEAD: <link rel="stylesheet" type="text/css" href="/styles/bigblow/css/htmlize.css"/>
5+
#+HTML_HEAD: <link rel="stylesheet" type="text/css" href="/styles/bigblow/css/bigblow.css"/>
6+
#+HTML_HEAD: <link rel="stylesheet" type="text/css" href="/styles/bigblow/css/hideshow.css"/>
77

8-
#+HTML_HEAD: <script type="text/javascript" src="docs/styles/bigblow/js/jquery-1.11.0.min.js"></script>
9-
#+HTML_HEAD: <script type="text/javascript" src="docs/styles/bigblow/js/jquery-ui-1.10.2.min.js"></script>
8+
#+HTML_HEAD: <script type="text/javascript" src="/styles/bigblow/js/jquery-1.11.0.min.js"></script>
9+
#+HTML_HEAD: <script type="text/javascript" src="/styles/bigblow/js/jquery-ui-1.10.2.min.js"></script>
1010

11-
#+HTML_HEAD: <script type="text/javascript" src="docs/styles/bigblow/js/jquery.localscroll-min.js"></script>
12-
#+HTML_HEAD: <script type="text/javascript" src="docs/styles/bigblow/js/jquery.scrollTo-1.4.3.1-min.js"></script>
13-
#+HTML_HEAD: <script type="text/javascript" src="docs/styles/bigblow/js/bigblow.js"></script>
14-
#+HTML_HEAD: <script type="text/javascript" src="docs/styles/bigblow/js/hideshow.js"></script>
15-
#+HTML_HEAD: <script type="text/javascript" src="src/lib/js/jquery.stickytableheaders.min.js"></script>
11+
#+HTML_HEAD: <script type="text/javascript" src="/styles/bigblow/js/jquery.localscroll-min.js"></script>
12+
#+HTML_HEAD: <script type="text/javascript" src="/styles/bigblow/js/jquery.scrollTo-1.4.3.1-min.js"></script>
13+
#+HTML_HEAD: <script type="text/javascript" src="/styles/bigblow/js/bigblow.js"></script>
14+
#+HTML_HEAD: <script type="text/javascript" src="/styles/bigblow/js/hideshow.js"></script>
15+
#+HTML_HEAD: <script type="text/javascript" src="/styles/lib/js/jquery.stickytableheaders.min.js"></script>

docs/styles/setup-darksun.org

+11-11
Original file line numberDiff line numberDiff line change
@@ -1,15 +1,15 @@
11
# -*- mode: org; -*-
22

3-
#+HTML_HEAD: <link rel="stylesheet" type="text/css" href="/docs/styles/darksun/css/htmlize.css"/>
4-
#+HTML_HEAD: <link rel="stylesheet" type="text/css" href="/docs/styles/darksun/css/darksun.css"/>
5-
#+HTML_HEAD: <link rel="stylesheet" type="text/css" href="/docs/styles/darksun/css/hideshow.css"/>
3+
#+HTML_HEAD: <link rel="stylesheet" type="text/css" href="/styles/darksun/css/htmlize.css"/>
4+
#+HTML_HEAD: <link rel="stylesheet" type="text/css" href="/styles/darksun/css/darksun.css"/>
5+
#+HTML_HEAD: <link rel="stylesheet" type="text/css" href="/styles/darksun/css/hideshow.css"/>
66

7-
#+HTML_HEAD: <script type="text/javascript" src="/docs/styles/darksun/js/jquery-1.11.0.min.js"></script>
8-
#+HTML_HEAD: <script type="text/javascript" src="/docs/styles/darksun/js/jquery-ui-1.10.2.min.js"></script>
7+
#+HTML_HEAD: <script type="text/javascript" src="/styles/darksun/js/jquery-1.11.0.min.js"></script>
8+
#+HTML_HEAD: <script type="text/javascript" src="/styles/darksun/js/jquery-ui-1.10.2.min.js"></script>
99

10-
#+HTML_HEAD: <script type="text/javascript" src="/docs/styles/darksun/js/jquery.localscroll-min.js"></script>
11-
#+HTML_HEAD: <script type="text/javascript" src="/docs/styles/darksun/js/jquery.scrollTo-1.4.3.1-min.js"></script>
12-
#+HTML_HEAD: <script type="text/javascript" src="/docs/styles/darksun/js/jquery.zclip.min.js"></script>
13-
#+HTML_HEAD: <script type="text/javascript" src="/docs/styles/darksun/js/darksun.js"></script>
14-
#+HTML_HEAD: <script type="text/javascript" src="/docs/styles/darksun/js/hideshow.js"></script>
15-
#+HTML_HEAD: <script type="text/javascript" src="/docs/styles/darksun/lib/js/jquery.stickytableheaders.min.js"></script>
10+
#+HTML_HEAD: <script type="text/javascript" src="/styles/darksun/js/jquery.localscroll-min.js"></script>
11+
#+HTML_HEAD: <script type="text/javascript" src="/styles/darksun/js/jquery.scrollTo-1.4.3.1-min.js"></script>
12+
#+HTML_HEAD: <script type="text/javascript" src="/styles/darksun/js/jquery.zclip.min.js"></script>
13+
#+HTML_HEAD: <script type="text/javascript" src="/styles/darksun/js/darksun.js"></script>
14+
#+HTML_HEAD: <script type="text/javascript" src="/styles/darksun/js/hideshow.js"></script>
15+
#+HTML_HEAD: <script type="text/javascript" src="/styles/darksun/lib/js/jquery.stickytableheaders.min.js"></script>

docs/styles/setup-readtheorg.org

+6-6
Original file line numberDiff line numberDiff line change
@@ -1,10 +1,10 @@
11
# -*- mode: org; -*-
22

33
#+OPTIONS: html-style:nil
4-
#+HTML_HEAD: <link rel="stylesheet" type="text/css" href="/docs/styles/readtheorg/css/htmlize.css"/>
5-
#+HTML_HEAD: <link rel="stylesheet" type="text/css" href="/docs/styles/readtheorg/css/readtheorg.css"/>
4+
#+HTML_HEAD: <link rel="stylesheet" type="text/css" href="/styles/readtheorg/css/htmlize.css"/>
5+
#+HTML_HEAD: <link rel="stylesheet" type="text/css" href="/styles/readtheorg/css/readtheorg.css"/>
66

7-
#+HTML_HEAD: <script type="text/javascript" src="/docs/styles/lib/js/jquery.min.js"></script>
8-
#+HTML_HEAD: <script type="text/javascript" src="/docs/styles/lib/js/bootstrap.min.js"></script>
9-
#+HTML_HEAD: <script type="text/javascript" src="/docs/styles/lib/js/jquery.stickytableheaders.min.js"></script>
10-
#+HTML_HEAD: <script type="text/javascript" src="/docs/styles/readtheorg/js/readtheorg.js"></script>
7+
#+HTML_HEAD: <script type="text/javascript" src="/styles/lib/js/jquery.min.js"></script>
8+
#+HTML_HEAD: <script type="text/javascript" src="/styles/lib/js/bootstrap.min.js"></script>
9+
#+HTML_HEAD: <script type="text/javascript" src="/styles/lib/js/jquery.stickytableheaders.min.js"></script>
10+
#+HTML_HEAD: <script type="text/javascript" src="/styles/readtheorg/js/readtheorg.js"></script>

docs/wscript_build

+8
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
bld.smplpkg('WireCellDocs')
2+
3+
if 'org2html' in bld.env.DOCS:
4+
styles = bld.path.ant_glob('styles/**')
5+
bld.install_files(bld.env.DOCS_INSTALL_PATH,
6+
styles,
7+
cwd=bld.path,
8+
relative_trick=True)

iface/README.org

+2-1
Original file line numberDiff line numberDiff line change
@@ -83,5 +83,6 @@ optimization".
8383
** ...
8484

8585
Any interfaces not listed above, please see their header file in
86-
[[file:inc/WireCellIface/][inc/WireCellIface/]] for more information.
86+
~inc/WireCellIface/~ for more information.
87+
8788

img/docs/notes.org

+1-1
Original file line numberDiff line numberDiff line change
@@ -234,7 +234,7 @@ A per-slice blobs can be viewed:
234234
paraview test-pdsp-748-blobs.vtu
235235
#+END_EXAMPLE
236236

237-
[[file:x.pdf]]
237+
[[file:x.svg]]
238238

239239
On Mate desktop, paraview renders tool tips with white text on yellow
240240
background which is unreadable. [[https://community.slickedit.com/index.php/topic,14388.0.html][This solution]] works. Install ~qtconfig~, "Appearance" tab, "Tune Palette..." button "ToolTipBase" and "ToolTipText" under "Central color roles". Pick nicer color (I just made text black) and save.

test/docs/bats.org

-2
Original file line numberDiff line numberDiff line change
@@ -342,5 +342,3 @@ Output to the special file descriptor ~3~ will always lead to that output to the
342342
}
343343
#+end_example
344344
Please avoid using this except in special, temporary cases, as it leads to very "noisy" tests.
345-
346-

test/docs/cpp.org

+2
Original file line numberDiff line numberDiff line change
@@ -262,3 +262,5 @@ In principle the path in ~argv[0]~ may also be used to locate the top of the ~wi
262262
#+begin_tip
263263
The data repository also provides a set of known files for input.
264264
#+end_tip
265+
266+

test/docs/zoo.org

+1-1
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@ This document describes a subset of the tests.
44

55
* test-pdsp-simsn
66

7-
This [[file:../test/test-pdsp-simsn.bats][test file]] runs PDSP simulation and signal processing, produces some standard output plots, and assuring no significant changes. It was adapted from https://www.phy.bnl.gov/~yuhw/wct-ci/gen/
7+
This [[file:../test/test-pdsp-simsn-nfsp.bats][test file]] runs PDSP simulation and signal processing, produces some standard output plots, and assuring no significant changes. It was adapted from https://www.phy.bnl.gov/~yuhw/wct-ci/gen/
88

99
* test-wct-uboone-img
1010

util/docs/rectangles.org

+1-1
Original file line numberDiff line numberDiff line change
@@ -47,7 +47,7 @@ This can be seen in the figure below. The green elemental rectangle
4747
holding the data ~[b]~ and the yellow elemental rectangle holding ~[c]~
4848
overlap to produce a region with the combination ~{b,c}~.
4949

50-
[[file:test_rectangles_find.svg]]
50+
[[file:talks/test_rectangles_find.svg]]
5151

5252
To implement the 2D extension simply, each dimension is handled with
5353
independent 1D interval maps. This produces a dimensional hierarchy

0 commit comments

Comments
 (0)