Emphasize that queries can contain arbitrary code

[telotortium]

I'm currently using org-ql-search to create a pseudo org-agenda buffer. However, the query language doesn't seem to allow filtering the results based on the result of some predicate (like, for example org-agenda-skip-function does). Does it make sense to add such a parameter to org-ql-search (and also org-ql-block, once #79 is resolved)?

The alternative is to write custom code to iterate over org-ql-view-buffer and remove entries myself (and also remove Super Groups that are now empty after removing entries).

For reference, I would use it to filter some entries from the following Org agenda custom command:

(defun my-org-super-agenda-group-by-project-or-task-group (item)
  "Output the name of the parent headline of the current headline.

In order to ensure that tasks that are part of projects are sorted before loose
tasks (tasks not part of projects), the name of the parent headline is prefixed
with “P: ” if it contains a TODO keyword and “TG: ” (for “task group”)
otherwise."
  (org-super-agenda--when-with-marker-buffer
    (org-super-agenda--get-marker item)
    (let* ((parent-title)
           (parent-has-todo))
      (save-excursion
        (when (org-up-heading-safe)
          (setq parent-title (org-get-heading 'notags 'notodo))
          (setq parent-has-todo
                (member (nth 2 (org-heading-components)) org-todo-keywords-1))))
      (when parent-title
        (if parent-has-todo
            (format "P: %s" parent-title)
          (format "TG: %s" parent-title))))))

(defcustom my-org-agenda-active-days 14
  "Number of days in the past to search for active projects")

(cl-defun my-org-agenda-next-projects ()
  "Show agenda for NEXT steps in org-mode projects

Use `org-ql-search' to search for all NEXT steps for projects.  Show only the
NEXT steps that have a timestamp within the last `my-org-agenda-active-days'
days."
  (interactive)
  (org-ql-search
    (org-agenda-files)
    `(and
      (todo "NEXT")
      (not (tags "HOLD" "CANCELLED" "ARCHIVED"))
      (not (scheduled :from 1))
      (ts :from ,(- my-org-agenda-active-days)))
    :super-groups '((:auto-map my-org-super-agenda-group-by-project-or-task-group))
    :sort 'date
    :title (format
            "NEXT (grouped by parent, except scheduled for future, %d-day active)"
            my-org-agenda-active-days)))

(cl-defun my-org-agenda-next-projects-agenda-command (unused)
  "Wrap `my-org-agenda-next-projects' for `org-agenda'."
  (my-org-agenda-next-projects))

(add-to-list 'org-agenda-custom-commands
             ;; TODO: Once https://github.com/alphapapa/org-ql/issues/79 is
             ;; fixed, use `org-ql-block' to avoid rendering agenda in 2
             ;; buffers.
             ;;
             ;; TODO: Once skip functions are implemented (track
             ;; https://github.com/alphapapa/org-ql/issues/3), add back
             ;; `bh/skip-non-tasks'.
             `("n" "NEXT (active, grouped by parent, except scheduled for future)"
               ((my-org-agenda-next-projects-agenda-command ""))
               ((org-agenda-write-buffer-name
                 "NEXT (active, grouped by parent, except scheduled for future)")
                (org-agenda-exporter-settings
                  my-org-agenda-export-options))
               "~/Downloads/agenda-n-export.pdf"))

[alphapapa]

Hi Robert,

However, the query language doesn't seem to allow filtering the results based on the result of some predicate (like, for example org-agenda-skip-function does).

If I understand you correctly, org-ql already supports this. Note this description from the docs:

An org-ql query is a lisp form which may contain arbitrary lisp forms, as well as certain built-in predicates.

Since it may contain arbitrary Lisp forms, you may call any function you like. The only requirements are that the function should expect to be called with point on a heading, it should not move point (i.e. use save-excursion), and it should return nil or non-nil, depending on whether the entry matches.

I should probably explain that more clearly in the docs, because it's probably easy to get the impression that only the built-in predicates are allowed.

Looking at the code you provided, here are a few thoughts:

1. I'm not sure what the wrapper function is for. It's probably not necessary, or shouldn't be, anyway.
2. The my-org-super-agenda-group-by-project-or-task-group is fine, but it may not be necessary. You might be able to achieve something similar using the parent or ancestors predicates in org-ql. Or you might be able to write it as an org-super-agenda auto-group, e.g. something like:

(org-super-agenda--def-auto-group project-task "their project status"
  :key-form (org-super-agenda--when-with-marker-buffer (org-super-agenda--get-marker item)
              (save-excursion
                (when (org-up-heading-safe)
                  (if (nth 2 (org-heading-components))
                      (concat "P: " (org-get-heading t t))
                    (concat "TG: " (org-get-heading t t)))))))

Or, if your primary goal is to affect how items are sorted, you might be able to write a custom sorting function and pass it to org-ql.

Does any of that help?

[telotortium]

Hi @alphapapa this is all very useful - thanks for getting back so quickly.

The reason I'm using the wrapper function is that it appears that org-ql-block doesn't accept all the same keyword arguments as org-ql-search does, include :super-groups, :sort, and :title. Therefore, I make a wrapper function that org-agenda can invoke.

[alphapapa]

I made some improvements to the docs about query expressions. Please let me know if you think any other changes are needed. Thanks.

[telotortium]

That's better - thanks!