Skip to content
Permalink

Comparing changes

Choose two branches to see what’s changed or to start a new pull request. If you need to, you can also or learn more about diff comparisons.

Open a pull request

Create a new pull request by comparing changes across two branches. If you need to, you can also . Learn more about diff comparisons here.
base repository: arjun-menon/alteza
Failed to load repositories. Confirm that selected base ref is valid, then try again.
Loading
base: v0.8.0
Choose a base ref
...
head repository: arjun-menon/alteza
Failed to load repositories. Confirm that selected head ref is valid, then try again.
Loading
compare: master
Choose a head ref

Commits on Jul 28, 2024

  1. Add 2 sentences for the GitHub action in README

    arjun-menon committed Jul 28, 2024
    Copy the full SHA
    67f8cb0 View commit details
  2. Add new top ideas

    arjun-menon committed Jul 28, 2024
    Copy the full SHA
    5cfe9cc View commit details
  3. Add idea

    arjun-menon committed Jul 28, 2024
    Copy the full SHA
    9857374 View commit details
  4. Add idea

    arjun-menon committed Jul 28, 2024
    Copy the full SHA
    449837f View commit details
  5. Add idea

    arjun-menon committed Jul 28, 2024
    Copy the full SHA
    52c4ec6 View commit details
  6. Add idea

    arjun-menon committed Jul 28, 2024
    Copy the full SHA
    4469bf0 View commit details
  7. Add some TODOs in engine.py

    arjun-menon committed Jul 28, 2024
    Copy the full SHA
    95acdd0 View commit details
  8. Minor engine method rename

    arjun-menon committed Jul 28, 2024
    Copy the full SHA
    8b2d549 View commit details
  9. Small refactor of enigne.process()

    arjun-menon committed Jul 28, 2024
    Copy the full SHA
    d4cd3d1 View commit details
  10. Add idea in ideas.md

    arjun-menon authored Jul 28, 2024

    Verified

    This commit was created on GitHub.com and signed with GitHub’s verified signature.
    Copy the full SHA
    831b12e View commit details
  11. Add idea in ideas.md

    arjun-menon authored Jul 28, 2024

    Verified

    This commit was created on GitHub.com and signed with GitHub’s verified signature.
    Copy the full SHA
    e36d148 View commit details

Commits on Jul 30, 2024

  1. Add dev_tmux_session.sh

    arjun-menon committed Jul 30, 2024
    Copy the full SHA
    56f40d8 View commit details
  2. Add check-types-and-lint.sh

    arjun-menon committed Jul 30, 2024
    Copy the full SHA
    1b2488c View commit details
  3. Add idea in ideas.md

    arjun-menon committed Jul 30, 2024
    Copy the full SHA
    ba696a4 View commit details
  4. Do resetOutputDir before Fs analysis

    arjun-menon committed Jul 30, 2024
    Copy the full SHA
    c406334 View commit details
  5. Implement skip

    arjun-menon committed Jul 30, 2024
    Copy the full SHA
    6dfcd30 View commit details
  6. Add test_content for skip

    arjun-menon committed Jul 30, 2024
    Copy the full SHA
    b1bce8d View commit details
  7. Refactor skipNames

    arjun-menon committed Jul 30, 2024
    Copy the full SHA
    b17c2e5 View commit details
  8. Update ideas.md since skip has been implemented.

    arjun-menon committed Jul 30, 2024
    Copy the full SHA
    84f384f View commit details

Commits on Jul 31, 2024

  1. Add py.typed per PEP 561

    arjun-menon committed Jul 31, 2024
    Copy the full SHA
    dd36a47 View commit details

Commits on Aug 1, 2024

  1. Add ideas in ideas.md

    arjun-menon committed Aug 1, 2024
    Copy the full SHA
    acf1e36 View commit details
  2. Slight re-organization in engine.py

    arjun-menon committed Aug 1, 2024
    Copy the full SHA
    b6c538c View commit details
  3. Minor log change

    arjun-menon committed Aug 1, 2024
    Copy the full SHA
    9e37f0a View commit details
  4. Version 0.8.1

    arjun-menon committed Aug 1, 2024
    Copy the full SHA
    c19e583 View commit details
  5. Minor README update

    arjun-menon committed Aug 1, 2024
    Copy the full SHA
    5965b9b View commit details
  6. In engine.py, move Generate inside Engine

    arjun-menon committed Aug 1, 2024
    Copy the full SHA
    867e4bb View commit details

Commits on Aug 2, 2024

  1. Update README.md

    arjun-menon authored Aug 2, 2024

    Verified

    This commit was created on GitHub.com and signed with GitHub’s verified signature.
    Copy the full SHA
    e313b88 View commit details
  2. Update README.md (minor)

    arjun-menon authored Aug 2, 2024

    Verified

    This commit was created on GitHub.com and signed with GitHub’s verified signature.
    Copy the full SHA
    8c80ff9 View commit details
  3. Update README.md (minor)

    arjun-menon authored Aug 2, 2024

    Verified

    This commit was created on GitHub.com and signed with GitHub’s verified signature.
    Copy the full SHA
    0b63e88 View commit details
  4. Implementing a --watch flag...

    arjun-menon committed Aug 2, 2024
    Copy the full SHA
    dd45fa4 View commit details

Commits on Aug 3, 2024

  1. Add Ctrl+C sigal handler for --watch

    arjun-menon committed Aug 3, 2024
    Copy the full SHA
    80ff04e View commit details
  2. Add build_test.sh

    arjun-menon committed Aug 3, 2024
    Copy the full SHA
    071f72a View commit details
  3. Version 0.8.2

    arjun-menon committed Aug 3, 2024
    Copy the full SHA
    93f296b View commit details
  4. Refactor Engine into an object, impl. checkIgnorePaths, etc.

    arjun-menon committed Aug 3, 2024
    Copy the full SHA
    bdd8401 View commit details
  5. Implement --ignore, refactor Fs a little bit, etc.

    arjun-menon committed Aug 3, 2024
    Copy the full SHA
    4dbb8d7 View commit details
  6. Convert line endings with dos2unix

    arjun-menon committed Aug 3, 2024
    Copy the full SHA
    4a1256d View commit details
  7. Version 0.8.3

    arjun-menon committed Aug 3, 2024
    Copy the full SHA
    bff648d View commit details
  8. Fix bug in --ignore flag

    arjun-menon committed Aug 3, 2024
    Copy the full SHA
    a3aca76 View commit details
  9. Version 0.8.4

    arjun-menon committed Aug 3, 2024
    Copy the full SHA
    860a69e View commit details
  10. Fix bug in --ignore implementation

    arjun-menon committed Aug 3, 2024
    Copy the full SHA
    a117896 View commit details
  11. Bump pyright to version 1.1.374

    arjun-menon committed Aug 3, 2024
    Copy the full SHA
    2ea6ff7 View commit details
  12. Version 0.8.5

    arjun-menon committed Aug 3, 2024
    Copy the full SHA
    6901e21 View commit details
  13. Minor refactor of Fs.defaultShouldIgnore

    arjun-menon committed Aug 3, 2024
    Copy the full SHA
    9f57dfa View commit details
  14. Add idea in ideas.md

    arjun-menon committed Aug 3, 2024
    Copy the full SHA
    f5614be View commit details
  15. Add one more condition for --ignore

    arjun-menon committed Aug 3, 2024
    Copy the full SHA
    fd842a7 View commit details

Commits on Aug 5, 2024

  1. Minor

    arjun-menon committed Aug 5, 2024
    Copy the full SHA
    1f922aa View commit details

Commits on Aug 6, 2024

  1. Turn getTitle() into a .title property

    arjun-menon committed Aug 6, 2024
    Copy the full SHA
    464a9f3 View commit details
  2. Turn getPages() into a .pages property

    arjun-menon committed Aug 6, 2024
    Copy the full SHA
    cf08515 View commit details
  3. Rework link(...) to use reflection

    arjun-menon committed Aug 6, 2024
    Copy the full SHA
    984e47b View commit details
  4. Update README with built-ins section

    arjun-menon committed Aug 6, 2024
    Copy the full SHA
    53b04d7 View commit details
2 changes: 1 addition & 1 deletion .github/workflows/type-checks.yml
Original file line number Diff line number Diff line change
@@ -44,7 +44,7 @@ jobs:
submodules: true

- name: Run Pyre
uses: facebook/pyre-action@60697a7858f7cc8470d8cc494a3cf2ad6b06560d
uses: facebook/pyre-action@v0.0.2
with:
# To customize these inputs:
# See https://github.com/facebook/pyre-action#inputs
238 changes: 224 additions & 14 deletions README.md
Original file line number Diff line number Diff line change
@@ -3,21 +3,23 @@
Alteza is a static site generator<sup>[<img height="10" width="10" src="https://upload.wikimedia.org/wikipedia/en/thumb/8/80/Wikipedia-logo-v2.svg/103px-Wikipedia-logo-v2.svg.png" />](https://en.wikipedia.org/wiki/Static_site_generator)</sup> driven by [PyPage](https://github.com/arjun-menon/pypage).
Examples of other static site generators can be [found here](https://github.com/collections/static-site-generators).

Alteza can be thought of as a simpler and more flexible alternative to static site generators like [Jekyll](https://jekyllrb.com/), [Hugo](https://gohugo.io/), [Zola](https://www.getzola.org/), [Nextra](https://nextra.site/), etc.

The differentiator with Alteza is that the site author (if familiar with Python) will have a lot more
fine-grained control over the output, than what (as far as I'm aware) any of the existing options offer.

The learning curve is also shorter with Alteza. I've tried to follow part of [xmonad](https://xmonad.org/)'s philosophy
The learning curve is also shorter with Alteza. I've tried to follow [xmonad](https://xmonad.org/)'s philosophy
of keeping things small and simple. Alteza doesn't try to do a lot of things; instead it simply offers the core crucial
functionality that is common to most static site generators.

Alteza also imposes very little _required_ structure or a particular "way of doing things" on your website (other than
requiring unique names). You retain the freedom to organize your website as you wish. (The name _Alteza_ comes from
a word that may be translated to [illustriousness](https://m.interglot.com/en/es/illustriousness) in Español.)
requiring unique names). You retain the freedom to organize your website as you wish. The name _Alteza_ comes from
a word that may be translated to [illustriousness](https://m.interglot.com/en/es/illustriousness) in Español.

A key design aspect of Alteza is writing little scripts and executing such code to generate your website. Your static
site can contain arbitrary Python that is executed at the time of site generation. [PyPage](https://github.com/arjun-menon/pypage),
in particular, makes it seamless to include actual Python code inside page templates. (This of course means that you
must run Alteza with trusted code, or in an isolated container. For example, in a [GitHub action](#github-action)–see instructions below.)
must run Alteza with trusted code, or in an isolated container. For example, in a [GitHub action–see instructions below](#github-action).)

## User Guide

@@ -28,6 +30,7 @@ must run Alteza with trusted code, or in an isolated container. For example, in
* Therefore, directories with no public files do not exist in the generated site.
* Files _reachable_ from marked-as-public files will also be publicly accessible.
* Here, reachability is discovered when a provided `link` function is used to link to other files.
* All files starting with a `.` are ignored.

3. All file and directory names, except for index page files, _at all depth levels_ must be unique. This is to simplify use of the `link(name)` function. With unique file and directory names, one can simply link to a file or directory with just its name, without needing to disambiguate a non-unique name with its path. Note: Directories can only be linked to if the directory contains an index page.

@@ -108,10 +111,201 @@ must run Alteza with trusted code, or in an isolated container. For example, in

2. Reachability of files is determined using this function, and unreachable files will be treated as non-public (and thus not exist in the generated site).

3. This function can be called both with a string identifying a file name, or with a reference to the file object itself. `link` will check the type of the argument passed to it, and appropriately handle each type.

4. This `link` function can also be called with string arguments using wiki-style links in Markdown files. For example, a `[[Happy Cat]]` in a Markdown file is the equivalent of writing `[Happy Cat]({{link('Happy Cat')}})`, or of writing `<a href="{{link('Happy Cat')}}">Happy Cat</a>`.

4. A file name's extension must be omitted while using `link` (including the `.py*` for any file with `.py` before its extension).
* i.e., e.g. one must write `link('magic-turtle')` for the file `magic-turtle.md`, and `link('pygments-styles')` for the file `pygments-styles.py.css`.
* Directories containing index files should just be referred to by the directory name. For example, the index page `about-me/hobbies/index.md` (or `about-me/hobbies/index.py.html`) should just be linked to with a `link('hobbies')`.

12. #### Expected (and Optional) Special Variables/Functions

Certain fields, with certain names, hold special meaning, and are called/used by Alteza. One such variable is `layout` (and `layoutRaw`), which points to the layout/template to be used to render the page (as explained in earlier points above). It can be overriden by descendant directories or pages.

#### Built-in Functions and Fields
<table>
<tr>
<th>Built-in</th>
<th>Description</th>
</tr>
<tr>
<td><code>link</code></td>
<td>

The `link` function takes **a name** or an object, and returns _a **relative** link_ to it. If a name is provided, it looks for that name in the NameRegistry (and throws an exception if the name wasn't found).

The `link` function has the side effect of making the linked-to page publicly accessible, if the page that is creating the link is reachable from another publicly-accessible page. The root `/` index page is always public.

_Note:_ for Markdown pages, an extra `../` is added at the beginning of the returned path to accommodate the fact that Markdown pages get turned into directories with the page rendered into an `index.html` inside the directory.

Availability:
<table>
<tr>
<td>Page</td>
<td>Template</td>
<td>Config</td>
<td>Index</td>
</tr>
<tr>
<td align="center">✅</td><td align="center">✅</td><td align="center">❌</td><td align="center">✅</td>
</tr>
</table>

</td>
</tr>
<tr>
<td><code>path</code></td>
<td>

The `path` function is similar to the `path` function above, except that:
* it _**does not**_ have the side effect of impacting the reachability graph, and making the linked-to page publicly accessible, and
* it also does not add an extra `../` at the beginning of the returned path for Markdown pages.

This function is good for use inside templates, to reference parent/ancestor templates for injection. For example, writing something like `{{ inject(path('skeleton')) }}`.

Available everywhere.

</td>
</tr>
<tr>
<td><code>dir</code></td>
<td>

The `dir` variables points to a `DirNode` object representing the directory that the relevant file is in.

This object has a fields like `dir.pages`, which is a list of all the pages (a list of `PageNode` objects) representing all the pages in that directory. Pages means Markdown files and HTML files. Some of the fields in `dir` are:

1. `dir.subDirs`: List of `FileNode` objects of files in this directory.
2. `dir.files`: List of `FileNode` objects of files in this directory.
3. `dir.pages`: List of `PageNode` objects of Markdown files, non-Markdown PyPage files, and HTML files.
4. `dir.indexPage`: A `PageNode` object of the index page, i.e. a `index.md` or a `index.html` file. If there is no index page, this is `None`.
5. `dir.title`: A string `title` object of the index page, only if the index page specifies a title. If there is no index page or no title specified by it, this is `None`.

In templates, the `dir` points to the directory that the file being processed is in.

Available everywhere.

</td>
</tr>
<tr>
<td>Title</td>
<td>The title is accessed with <code>page.title</code>. It is picked up either from PyPage code in the page or a <code>title</code> YAML field in the file. If `title` is not defined by the page, then <code>page.realName</code> of the file is used, which is the adjusted name of the file without its extension and idea date prefix (if present) removed. The title isn't <em>properly</em> available to Python inside the page itself, or from <code>__config__.py</code>, since the page has not been processed when these are executed. If <code>page.title</code> is accessed from these (the page or config), or if a <code>title</code> was never defined in the page, then the <code>.realName</code> of the file would be returned.

Note: the title can directly be accessed as `title` (without `pageObj.title`) in the template (and [inherited](https://github.com/arjun-menon/pypage?tab=readme-ov-file#inheritance-with-inject-and-exists) templates) for the page, since all environment variables from the page are passed on to the template, during template processing.

Availability:
<table>
<tr>
<td>Page</td>
<td>Template</td>
<td>Config</td>
<td>Index</td>
</tr>
<tr>
<td align="center">❌</td><td align="center">✅</td><td align="center">❌</td><td align="center">✅</td>
</tr>
</table>

</td>
</tr>
<tr>
<td>YAML fields & other vars</td>
<td>

YAML fields (and other variables defined in PyPage code) of a page are:
* Available directly to template(s) that the page uses/invokes.
* Stored in `pageObj.env`, for future access. The index page, for example, can use `page.env` to access these fields & variables.
* Stored _**as attributes**_ in the `PyPageNode` page object, as long as the `env` var does not conflict with an existing attribute of `PyPageNode`.
* This enables referring to a field or variable with just `page.fieldName` (instead of having to write `page.env[fieldName]`, which is also valid).
<br />

Availability (same as `title`):
<table>
<tr>
<td>Page</td>
<td>Template</td>
<td>Config</td>
<td>Index</td>
</tr>
<tr>
<td align="center">❌</td><td align="center">✅</td><td align="center">❌</td><td align="center">✅</td>
</tr>
</table>

</td>
</tr>
<tr>
<td>Last Modified Date & Time</td>
<td>

_This is only available on `PageNode` objects._

The last modified date & time for a given file is taken from:

a. The date & time of _the last commit that modified that file_, in git history, if the file is inside a git repo.

b. The last modified date & time as provided by the file system.

There's a `getLastModifiedObj()` function which returns a Python `datetime` object. There's also a `getLastModified(f: str = default_datetime_format)` functon which returns a `str` with the date & time formatted.

The `default_datetime_format` is `%Y %b %-d at %-H:%M %p`.

_Note:_ This function calls spawns a `git` process, so is a tiny bit slow.

Available everywhere.

</td>
</tr>
<tr>
<td>Idea Date</td>
<td>

_This is only available on `PageNode` objects._

The "idea date" for a given file is either:

a. For a Markdown file, a date prefix before the markdown file's name, in the form `YYYY-MM-DD`.

b. If not a Markdown file or there's no date prefix, and _the file is in a git repo_, then the idea date is the date of the first commit that introduced the file into git history. (Note: this breaks if the file was renamed or moved.)

c. If there is neither a date prefix and the file is not in a git repo, there is no idea date for that file (i.e. it's `None` or `""`).

There's a `getIdeaDateObj()` function which returns a Python `date` object (or `None` if there's no idea). There's also a `getIdeaDate(f: str = default_date_format)` functon which returns a `str` with the date & time formatted or `""` if there's no idea date.

The `default_date_format` is `%Y %b %-d`.

_Note:_ This function calls spawns a `git` process, if it's not a Markdown file or if there is no date prefix in the Markdown file's name.

Available everywhere.

</td>
</tr>
<tr>
<td><code>readfile</code></td>
<td>This is just a simple built-in function that reads the contents of a file (assuming <code>utf-8</code> encoding) into a string, and returns it.
Available everywhere.
</td>
</tr>

<tr>
<td><code>sh</code></td>
<td>This exposes the entire <code>sh</code> library. The current working directory (CWD) would be wherever the file being executed is located (regardless of whether the file is a regular page or index page or <code>__config__.py</code> or template). If the file is a template, the CWD would be that of the page being processed.

See `sh`'s documentation here: https://sh.readthedocs.io/en/latest/

Available everywhere.
</td>
</tr>

<tr>
<td><code>skip</code></td>
<td>This environment variable, if specified, is a list of names of files or directories to be skipped. (It must be of type <code>List[str]</code>, if defined.)
</td>
</tr>

</table>

## GitHub Action, Installation & Command-Line Usage

### GitHub Action
@@ -144,10 +338,12 @@ jobs:
steps:
- name: Generate Alteza Website
id: generate
uses: arjun-menon/alteza@v0.8.0
uses: arjun-menon/alteza@v0.9.1
with:
path: .
```
The last parameter `path` should specify which directory in your GitHub repo should be rendered into a website. Also, note: make sure to set the `branches` for `workflow_dispatch` correctly (to your branch) so that this action is triggered on each push.

For an example of this GitHub workflow above in action, see [alteza-test](https://github.com/arjun-menon/alteza-test) ([yaml](https://github.com/arjun-menon/alteza-test/blob/main/.github/workflows/alteza.yml), [runs](https://github.com/arjun-menon/alteza-test/actions/workflows/alteza.yml)).

### Installation
@@ -170,17 +366,31 @@ If you're working on Alteza itself, then run the `alteza` module itself, from th
### Command-line Arguments
The `-h` argument above will print the list of available arguments:
```
usage: __main__.py --content CONTENT --output OUTPUT [--clear_output_dir] [--copy_assets] [--seed SEED] [-h]
usage: __main__.py --content CONTENT --output OUTPUT [--clear_output_dir] [--copy_assets] [--seed SEED] [--watch]
[--ignore [IGNORE ...]] [-h]
options:
--content CONTENT (str, required) Directory to read the input content from.
--output OUTPUT (str, required) Directory to send the output to. WARNING: This will be deleted.
--clear_output_dir (bool, default=False) Delete the output directory, if it already exists.
--copy_assets (bool, default=False) Copy static assets instead of symlinking to them.
--seed SEED (str, default={}) Seed JSON data to add to the initial root env.
-h, --help show this help message and exit
--content CONTENT (str, required) Directory to read the input content from.
--output OUTPUT (str, required) Directory to write the generated site to.
--clear_output_dir (bool, default=False) Delete the output directory, if it already exists.
--copy_assets (bool, default=False) Copy static assets instead of symlinking to them.
--seed SEED (str, default={}) Seed JSON data to add to the initial root env.
--watch (bool, default=False) Watch for content changes, and rebuild.
--ignore [IGNORE ...]
(List[str], default=[]) Paths to completely ignore.
-h, --help show this help message and exit
```
As might be obvious above, you set the `content` to your content directory. The output directory will be deleted entirely, before being written to.
As might be obvious above, you set the `--content` field to your content directory.

The output directory for the generated site is specified with `--output`. You can have Alteza automatically delete it entirely before being written to (including in `--watch` mode) by setting the `--clear_output_dir` flag.

Normally, Alteza performs a single build and exits. With the `--watch` flag, Alteza monitors the file system for changes, and rebuilds the site automatically.

The `--ignore` flag is a list of _paths_ to files or directories to ignore. This is useful for ignoring directories like `.gitignore`, or other non-pertinent files and directories.

Normal Alteza behavior for static assets is to create symlinks from your generate site to static files in your content directory. You can turn off this behavior with `--copy_assets`.

The `--seed` flag is a JSON string representing seed data for PyPage processing. This seed is injected into every PyPage document. The seed _is not global_, and so cannot be modified between files; it is copied into each PyPage execution environment.

To test against `test_content` (and generate output to `test_output`), run it like this:
```sh
@@ -221,7 +431,7 @@ To run it along with all the type checks (excluding `pytype`), just run: `mypy a

Of course, when it makes sense, lints should be suppressed next to the relevant line, in code. Also, unlike typical Python code, the naming convention generally-followed in this codebase is `camelCase`. Pylint checks for names have mostly been disabled.

Here's the Pylint-generated UML diagram of Alteza's code (that's current as of v0.8.0):
Here's the Pylint-generated UML diagram of Alteza's code (that's current as of v0.9.0):

![](https://raw.githubusercontent.com/arjun-menon/alteza/master/uml-diagram.png)

2 changes: 1 addition & 1 deletion action.yml
Original file line number Diff line number Diff line change
@@ -38,7 +38,7 @@ runs:
# release that is limited to documentation and `action.yml`.
#
# pip install -q git+https://github.com/arjun-menon/alteza.git@master
pip install -q alteza==0.8.0
pip install -q alteza==0.9.1
- name: Generate
shell: bash
4 changes: 2 additions & 2 deletions alteza/__main__.py
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
#!/usr/bin/env python3
from .engine import run, Args
from .engine import Engine, Args


def main() -> None:
run(Args().parse_args())
Engine(Args().parse_args()).run()


# See: https://chriswarrick.com/blog/2014/09/15/python-apps-the-right-way-entry_points-and-scripts/
377 changes: 266 additions & 111 deletions alteza/engine.py

Large diffs are not rendered by default.

171 changes: 115 additions & 56 deletions alteza/fs.py
Original file line number Diff line number Diff line change
@@ -2,13 +2,15 @@
import os
import re
from collections import defaultdict
from dataclasses import dataclass
from datetime import date, datetime
from subprocess import check_output, CalledProcessError, STDOUT
from typing import (
Optional,
Any,
List,
Sequence,
Iterator,
Dict,
DefaultDict,
Set,
@@ -20,6 +22,7 @@
import markdown
import yaml
from colored import Style, Fore # type: ignore
from markdown.extensions.wikilinks import WikiLinkExtension

colored_logs = True

@@ -54,13 +57,24 @@ def setNodeAsPublic(self) -> None:
self.shouldPublish = True

def makePublic(self) -> None:
Fs.runOnFsNodeAndAscendantNodes(self, lambda fsNode: fsNode.setNodeAsPublic())
self.runOnFsNodeAndAscendantNodes(self, lambda fsNode: fsNode.setNodeAsPublic())

def isParentGitRepo(self) -> bool:
if self.parent is None:
return DirNode.isPwdGitRepo()
return self.parent.isInGitRepo

@staticmethod
def runOnFsNodeAndAscendantNodes(
startingNode: "FsNode", fn: Callable[["FsNode"], None]
) -> None:
def walk(node: FsNode) -> None:
fn(node)
if node.parent is not None:
walk(node.parent)

walk(startingNode)


class FileNode(FsNode):
@staticmethod
@@ -118,7 +132,8 @@ def getLinkName(self) -> str:
return self.getParentDir().getRectifiedName()
return self.realName

def getTitle(self) -> str:
@property
def title(self) -> str:
if "title" in self.env:
return self.env["title"]
return self.realName
@@ -152,8 +167,8 @@ def __init__(
self,
parent: Optional["DirNode"],
dirPath: str,
# shouldIgnore(name: str, isDir: bool) -> bool
shouldIgnore: Callable[[str, bool], bool],
# shouldIgnore(name: str, parentPath: str, isDir: bool) -> bool
shouldIgnore: Callable[[str, str, bool], bool],
) -> None:
_, subDirNames, fileNames = next(os.walk(dirPath))
dirPath = "" if dirPath == os.curdir else dirPath
@@ -162,12 +177,12 @@ def __init__(
self.files: List[FileNode] = [
FileNode.construct(self, dirPath, fileName)
for fileName in fileNames
if not shouldIgnore(fileName, False)
if not shouldIgnore(fileName, self.fullPath, False)
]
self.subDirs: List[DirNode] = [
DirNode(self, os.path.join(dirPath, subDirName), shouldIgnore)
for subDirName in subDirNames
if not shouldIgnore(subDirName, True)
if not shouldIgnore(subDirName, self.fullPath, True)
]

def getRectifiedName(self) -> str:
@@ -188,9 +203,30 @@ def isPwdGitRepo() -> bool:
except CalledProcessError:
return False

def getPages(self) -> Sequence["PageNode"]:
@property
def pages(self) -> Sequence["PageNode"]:
return [f for f in self.files if (isinstance(f, PageNode) and not f.isIndex())]

def getPyPagesOtherThanIndex(self) -> Iterator["PyPageNode"]:
return (
f for f in self.files if (isinstance(f, PyPageNode) and not f.isIndex())
)

@functools.cached_property
def indexPage(self) -> Optional["PageNode"]:
indexFilter = filter(lambda f: f.isIndex(), self.files)
indexFile: Optional[FileNode] = next(indexFilter, None)
if indexFile:
assert isinstance(indexFile, PageNode)
return indexFile
return None

@property
def title(self) -> Optional[str]:
if self.indexPage and "title" in self.indexPage.env:
return self.indexPage.title
return None

@staticmethod
def _displayDir(dirNode: "DirNode", indent: int = 0) -> str:
return (
@@ -261,23 +297,63 @@ def gitFirstAuthDate(self) -> Optional[date]:
return None
return None

def __getattr__(self, attr: str) -> None:
"""Allows for checking whether page.some_property exists more easily (without `hasattr`)."""
return None


class PyPageNode(PageNode):
temporal_link: Optional[Callable[[str], str]] = None

def __init__(self, parent: Optional[DirNode], dirPath: str, fileName: str) -> None:
super().__init__(parent, dirPath, fileName)
self._pyPageOutput: Optional[str] = None # to be generated (by pypage)

def setPyPageOutput(self, output: str) -> None:
self._pyPageOutput = output

def getPyPageOutput(self) -> str:
@property
def output(self) -> str:
if self._pyPageOutput is None:
raise AltezaException("PyPage output has not been generated yet.")
assert isinstance(self._pyPageOutput, str)
return self._pyPageOutput

@output.setter
def output(self, htmlOutput: str) -> None:
self._pyPageOutput = htmlOutput


def buildWikiUrl(label: str, base: str, end: str) -> str:
# pylint: disable=unused-argument
if PyPageNode.temporal_link is None:
raise AltezaException("PyPageNode.temporal_link is not set.")
# pylint: disable=not-callable
return PyPageNode.temporal_link(label)


class Md(PyPageNode):
md = markdown.Markdown(
# See: https://python-markdown.github.io/extensions/
extensions=[
# Extra extensions:
"abbr",
"attr_list",
"def_list",
"fenced_code",
"footnotes",
"md_in_html",
"tables",
# Standard extensions:
"admonition",
"codehilite",
"meta",
"mdx_breakless_lists",
# "sane_lists",
"mdx_truly_sane_lists",
"smarty", # not sure
"toc",
WikiLinkExtension(html_class="", build_url=buildWikiUrl),
]
)

def __init__(self, parent: Optional[DirNode], dirPath: str, fileName: str) -> None:
super().__init__(parent, dirPath, fileName)

@@ -298,30 +374,10 @@ class Result(NamedTuple):

@staticmethod
def processMarkdown(text: str) -> Result:
md = markdown.Markdown(
# See: https://python-markdown.github.io/extensions/
extensions=[
# Extra extensions:
"abbr",
"attr_list",
"def_list",
"fenced_code",
"footnotes",
"md_in_html",
"tables",
# Standard extensions:
"admonition",
"codehilite",
"meta",
"sane_lists",
"smarty", # not sure
"toc",
]
)
html: str = md.convert(text)
html: str = Md.md.convert(text)
yamlFrontMatter: str = ""

for name, lines in md.Meta.items(): # type: ignore # pylint: disable=no-member
for name, lines in Md.md.Meta.items(): # type: ignore # pylint: disable=no-member
yamlFrontMatter += f"{name} : {lines[0]} \n"
for line in lines[1:]:
yamlFrontMatter += " " * (len(name) + 3) + line + "\n"
@@ -337,7 +393,7 @@ def processMarkdown(text: str) -> Result:

class NonMd(PyPageNode):
def __init__(
# pylint: disable=too-many-arguments
# pylint: disable=too-many-arguments, too-many-positional-arguments
self,
realName: str,
rectifiedFileName: str,
@@ -395,32 +451,27 @@ def __repr__(self) -> str:
)


@dataclass
class FsCrawlResult:
rootDir: DirNode
nameRegistry: NameRegistry


class Fs:
configFileName: str = "__config__.py"
ignoreAbsPaths: List[str] = []

@staticmethod
def readfile(file_path: str) -> str:
with open(file_path, "r", encoding="utf-8") as someFile:
return someFile.read()

@staticmethod
def runOnFsNodeAndAscendantNodes(
startingNode: FsNode, fn: Callable[[FsNode], None]
) -> None:
def walk(node: FsNode) -> None:
fn(node)
if node.parent is not None:
walk(node.parent)

walk(startingNode)

@staticmethod
def isHidden(name: str) -> bool:
return name.startswith(".")

@staticmethod
def defaultShouldIgnore(name: str, isDir: bool) -> bool:
# pylint: disable=unused-argument
def shouldIgnoreStandard(name: str) -> bool:
if Fs.isHidden(name):
return True
if name in {"__pycache__"}:
@@ -432,6 +483,19 @@ def defaultShouldIgnore(name: str, isDir: bool) -> bool:
return True
return False

@staticmethod
def defaultShouldIgnore(name: str, parentPath: str, isDir: bool) -> bool:
# pylint: disable=unused-argument
if Fs.shouldIgnoreStandard(name):
return True

fullPath = os.path.abspath(os.path.join(parentPath, name))
for ignoreAbsPath in Fs.ignoreAbsPaths:
if ignoreAbsPath in fullPath:
return True

return False

@staticmethod
def defaultSkipForRegistry(name: str) -> bool:
if name == Fs.configFileName:
@@ -440,10 +504,10 @@ def defaultSkipForRegistry(name: str) -> bool:

@staticmethod
def crawl(
# Signature -- shouldIgnore(name: str, isDir: bool) -> bool
shouldIgnore: Callable[[str, bool], bool] = defaultShouldIgnore,
# Signature -- shouldIgnore(name: str, parentPath: str, isDir: bool) -> bool
shouldIgnore: Callable[[str, str, bool], bool] = defaultShouldIgnore,
skipForRegistry: Callable[[str], bool] = defaultSkipForRegistry,
) -> Tuple[DirNode, NameRegistry]:
) -> FsCrawlResult:
"""
Crawl the current directory. Construct & return an FsNode tree and NameRegistry.
"""
@@ -452,9 +516,4 @@ def crawl(
rootDir: DirNode = DirNode(None, dirPath, shouldIgnore)
nameRegistry = NameRegistry(rootDir, skipForRegistry)

return rootDir, nameRegistry

def __init__(self) -> None:
rootDir, nameRegistry = Fs.crawl()
self.rootDir: DirNode = rootDir
self.nameRegistry: NameRegistry = nameRegistry
return FsCrawlResult(rootDir, nameRegistry)
Empty file added alteza/py.typed
Empty file.
10 changes: 10 additions & 0 deletions alteza/version.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
version: str = "0.9.1"

name: str = "alteza"

# pylint: disable=consider-using-f-string
repo_url: str = "https://github.com/arjun-menon/%s" % name
download_url: str = "%s/archive/v%s.tar.gz" % (repo_url, version)

author: str = "Arjun G. Menon"
author_email: str = "contact@arjungmenon.com"
1 change: 1 addition & 0 deletions build_test.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
python -m alteza --content test_content --output test_output --clear_output_dir --watch --ignore test_content/dirA
1 change: 1 addition & 0 deletions check-types-and-lint.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
mypy alteza ; pyre check ; pyright alteza ; pyflakes alteza ; pylint -j 0 alteza
29 changes: 29 additions & 0 deletions dev_tmux_session.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
#!/usr/bin/env bash

SESSION_NAME="alteza_work_session"

mkdir -p test_output/

# Check if the session already exists
if tmux has-session -t $SESSION_NAME 2>/dev/null; then
echo "Session $SESSION_NAME already exists. Attaching to it."
tmux attach-session -t $SESSION_NAME
else
# Create a new session and name it
tmux new-session -d -s $SESSION_NAME

# Split the window horizontally
tmux split-window -v -l 20%

# Send a command to the first pane
tmux send-keys -t 0 'cd test_output; python3 -m http.server 1234' C-m

# Send a command to the second pane
tmux send-keys -t 1 'source venv/bin/activate.fish' C-m

tmux swap-pane -s 0 -t 1

# Attach to the created session
tmux attach-session -t $SESSION_NAME
fi

39 changes: 38 additions & 1 deletion ideas.md
Original file line number Diff line number Diff line change
@@ -3,7 +3,41 @@ Flexible Static Site Generator
------------------------------

#### Ideas...
* Next up:
* Top 1:
* Proper object exposed for each PyPage file.
* Proper YAML field capture.
* Maybe inject post-processing fields (gathered with `getModuleVars`) into this object with `setattr`.
* Document all built-in functions, and config fields (like the `skip` config var), etc.
* Update the `YYYY-MM-DD-` handler to allow `YYYY-MM-DD ` as well.
* A `nameEncode` configurable function, with a `defaultNameEncode`.
* Convert something like "Purpose of Life" to `purpose-of-life`.
* This will allow us to use the file name as title, like in Obsidian.
* Use a regex & validate that what's returned by `nameEncode` is acceptable for a URL.
* Document this behavior: with Name Registry, this will make the unique names rule non-case-sensitive, at least with `defaultNameEncode` converting all chars to lower case.
* Rename `subDirs` to `dirs`, etc.?
* An `after` function defined in __config__.py that gets run after all children are processed.
* Installing pip requirements.txt for the site being built.
* Maybe: Expose `content` for user post-processing functions.
* Avoid rebuilding unchanged directories while re-building with --watch, unless any ascendant __config__.py has changed.

**_Completed_**:
- [x] Rebuild automatically with a fs watching library.
- [x] Implement a `skip` config var.
- [x] Document most built-in functions.
- [x] Obsidian style Wiki Links.

---

* Top 2:
* A `--dev` flag with supports auto-refresh, using an approach similar to: https://github.com/baalimago/wd-41/blob/main/internal/wsinject/delta_streamer.ws.go
* A `--zip` flag that create a zip file of the outputted website, with a special `_raw` directory in there containing the raw source code of the website as well (or maybe that should be a separate option, or sub-option).
* Also, the `--dev` flag serve the site. E.g. see: https://stackoverflow.com/questions/33028624/run-python-httpserver-in-background-and-continue-script-execution
* Add a trailing `/` slash for Markdown page dirs since: (a) if a Markdown page is turned into a dir/collection of smaller essays, this would allow that change to happen naturally/seamlessly, and (b) since many web servers including the one used by GH pages add a trailing `/` slash to directories using a 301 Redirect anyways.
* Caching in the GitHub action. See:
* https://docs.github.com/en/actions/using-workflows/caching-dependencies-to-speed-up-workflows
* https://github.com/actions/setup-python#caching-packages-dependencies
* https://jcdan3.medium.com/4-ways-to-speed-up-your-github-action-workflows-a0b08067a6c6
* Enforce directory name uniqueness--currently multiple directories that are all without index pages can share the same name.
* Rectify `linkName` (if necessary).
* In addition to the `files` field, we'll need some other fields.
* Add `pages` (for Md & HTML), and `pyPages` (for all pypages).
@@ -73,6 +107,9 @@ Flexible Static Site Generator
* ~~The Markdown file is processed using `pypage`, with its Python environment enhanced by the YAML fields from the front matter~~.
* The environment dictionary after the Markdown is processed by pypage is treated as the "return value" of this `.md` file. ~~This "return value" dictionary has a `content` key added to it which maps to the `pypage` output for this `.md` file~~.
* In the GitHub action: an option to pip install a `requirements.txt` file. (It'd just do `touch requirements.txt; pip install requirements.txt`.) Or maybe not an option, but always-on behavior.
* Archiving URLs.
* See: https://gwern.net/archiving
* Perhaps with https://github.com/oduwsdl/archivenow

Older Ideas 2
=============
4 changes: 2 additions & 2 deletions requirements-dev.txt
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
black~=24.4.2
pylint
pylint~=3.3.1
pip-tools
mypy ~= 1.11.0
pyre-check == 0.9.19
pyright ~= 1.1.372
pyright ~= 1.1.374
pytype ~= 2023.12.18
pyflakes ~= 3.2.0
types-PyYAML >= 6.0.12.12
11 changes: 10 additions & 1 deletion requirements.txt
Original file line number Diff line number Diff line change
@@ -9,14 +9,21 @@ colored==2.2.4
docstring-parser==0.15
# via typed-argument-parser
markdown==3.6
# via
# alteza (setup.py)
# mdx-breakless-lists
# mdx-truly-sane-lists
mdx-breakless-lists==1.0.1
# via alteza (setup.py)
mdx-truly-sane-lists==1.3
# via alteza (setup.py)
mypy-extensions==1.0.0
# via typing-inspect
packaging==24.1
# via typed-argument-parser
pygments==2.17.2
# via alteza (setup.py)
pypage==2.0.9
pypage==2.1.0
# via alteza (setup.py)
pyyaml==6.0.1
# via alteza (setup.py)
@@ -28,3 +35,5 @@ typing-extensions==4.9.0
# via typing-inspect
typing-inspect==0.9.0
# via typed-argument-parser
watchdog==4.0.1
# via alteza (setup.py)
16 changes: 7 additions & 9 deletions setup.py
Original file line number Diff line number Diff line change
@@ -1,11 +1,6 @@
from setuptools import setup

version = "0.8.0"

name = "alteza"

repo_url = "https://github.com/arjun-menon/%s" % name
download_url = "%s/archive/v%s.tar.gz" % (repo_url, version)
from alteza.version import version, name, repo_url, download_url, author, author_email

setup(
name=name,
@@ -15,20 +10,23 @@
# Ref: https://packaging.python.org/en/latest/discussions/install-requires-vs-requirements/
install_requires=[
# Standard dependencies:
"pypage >= 2.0.9",
"pypage >= 2.1.0",
"markdown >= 3.6",
"mdx-breakless-lists >= 1.0.1",
"mdx_truly_sane_lists >= 1.3",
"pygments >= 2.16.1",
"pyyaml >= 6.0.1",
"colored >= 2.2.4",
"sh >= 2.0.7",
"typed-argument-parser >= 1.10.1",
"watchdog >= 4.0.1",
],
long_description=open("README.md").read(),
long_description_content_type="text/markdown",
url=repo_url,
download_url=download_url,
author="Arjun G. Menon",
author_email="contact@arjungmenon.com",
author=author,
author_email=author_email,
keywords=["static site generator", "static sites", "ssg"],
license="AGPL-3.0-or-later",
# Ref:https://docs.python.org/3.11/distutils/examples.html
4 changes: 4 additions & 0 deletions test_content/index.py.html
Original file line number Diff line number Diff line change
@@ -2,3 +2,7 @@
Go to <a href="{{ link('sectionM') }}">Section M</a>...
<br />
Or, <a href="{{ link('sectionL') }}">Section L</a>.

<br />
<br />
Less interesting: <a href="{{ link('sectionK') }}">Section K</a>.
17 changes: 17 additions & 0 deletions test_content/sectionK/index.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
title: Just Section K
---

<p>
The directories here are:
<ol>
{% for d in dir.subDirs %}
{% if d.title %}
<li>
<a href="{{link(d)}}">{{d.title}}</a>
</li>
{% endif %}
{% endfor %}
</ol>
</p>

<a href="..">Root/home</a>.
2 changes: 1 addition & 1 deletion test_content/sectionK/sectionL/index.md
Original file line number Diff line number Diff line change
@@ -6,7 +6,7 @@ Welcome to Section L
{{
for file in dir.files:
write('<li><a href="%s">%s</a> <code>%s</code> </li>' %
(linkObj(file), file.getTitle(), ' -> ' + file.fullPath))
(link(file), file.title, ' -> ' + file.fullPath))
}}
</ol>

10 changes: 7 additions & 3 deletions test_content/sectionK/sectionM/index.py.html
Original file line number Diff line number Diff line change
@@ -17,11 +17,15 @@ <h4>
</h4>

<ul>
{% for file in dir.getPages() %}
{% for file in dir.pages %}
<li>
<a href="{{ linkObj(file) }}">
{{file.getTitle()}}
<a href="{{ link(file) }}">
{{file.title}}
</a>
{{
if file.x1y:
write('&nbsp; &nbsp;' + file.x1y)
}}
</li>
{% endfor %}
</ul>
5 changes: 5 additions & 0 deletions test_content/sectionK/sectionM/joyful-squirrel.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
title: {{ title + " about a Squirrel" }}
x1y: :-)
---

Here's a joyful squirrel.
@@ -7,10 +8,14 @@ Here's a joyful squirrel.

Want to learn about a [magic turtle]({{link("magic-turtle")}})?

What about a [[curious-cat]]?

This is [what a "megabyte" is]({{link("just_a_test")}}).

This page was started on {{getIdeaDate("%B %-d, %Y")}}.
* We got this from git history.

This page was last modified on {{getLastModified()}}.
- We got this from git history as well.

Back [home](..).
1 change: 1 addition & 0 deletions test_content/sectionY/test_skip/__config__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
skip = ["fast_rocket", "flowery_garden"]
3 changes: 3 additions & 0 deletions test_content/sectionY/test_skip/fast_rocket.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
public: true
---
Is there a rocket that's slow?
3 changes: 3 additions & 0 deletions test_content/sectionY/test_skip/other_skip/flowery_garden.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
public: true
---
A flowery garden.
Binary file modified uml-diagram.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.