Skip to content

Commit f4918b1

Browse files
committed
Merge: wip/helm-query-rewriting
It's finally time to merge this branch, which adds non-sexp queries, updates the readme, and a few other things. Squashed commit of the following: commit ee54e31 Author: Adam Porter <[email protected]> Date: Sun Oct 6 00:10:19 2019 -0500 WIP: Depend-on/Require dash-functional commit 42f6a47 Author: Adam Porter <[email protected]> Date: Sat Oct 5 23:45:06 2019 -0500 WIP: Update docs commit eb56e46 Author: Adam Porter <[email protected]> Date: Sat Oct 5 22:04:24 2019 -0500 WIP: Add demo script commit cdd394f Author: Adam Porter <[email protected]> Date: Sat Oct 5 20:24:44 2019 -0500 WIP: Update demo GIF commit 7f470a7 Author: Adam Porter <[email protected]> Date: Sat Oct 5 13:02:09 2019 -0500 WIP: (property) Convert keyword properties to strings I'm a little uncomfortable with how query parsing and processing is getting spread out among 2 or 3 functions, but handling this in the plain-query parsing function would require uglier special-case workarounds, so this seems like the best solution for now. commit 4e494a0 Author: Adam Porter <[email protected]> Date: Fri Oct 4 16:25:00 2019 -0500 WIP: Handle query errors, move code, add tags-all, tags& aliases This prevents query compilation errors from displaying a warning buffer, especially in the Helm command. Close to done, but now need to figure out the "property:" predicate, because its value argument looks like an alternative property rather than a value, e.g. "property:agenda-group,plans" in which "plans" is the value for the property "agenda-group". commit 9d8222d Author: Adam Porter <[email protected]> Date: Thu Oct 3 22:18:29 2019 -0500 WIP: Rewrite (priority) predicate to accept multiple priorities This commit also fixes a latent bug in the preamble for (priority), which wasn't being hit because the query (priority) was being rewritten by pre-process to (priority '>= C). commit 4900470 Author: Adam Porter <[email protected]> Date: Thu Oct 3 20:41:54 2019 -0500 WIP: More documentation commit 5041a90 Author: Adam Porter <[email protected]> Date: Thu Oct 3 12:30:50 2019 -0500 WIP: Mention query types for commands commit bb2d185 Author: Adam Porter <[email protected]> Date: Thu Oct 3 11:01:56 2019 -0500 WIP: Remove old code commit 8de048e Author: Adam Porter <[email protected]> Date: Thu Oct 3 11:01:03 2019 -0500 WIP: TODO done commit 6a38fc8 Author: Adam Porter <[email protected]> Date: Thu Oct 3 10:59:45 2019 -0500 WIP: Tidy docstrings commit 04d0c25 Author: Adam Porter <[email protected]> Date: Thu Oct 3 10:58:21 2019 -0500 WIP: Document plain queries commit 23dcc9b Author: Adam Porter <[email protected]> Date: Thu Oct 3 10:27:41 2019 -0500 WIP: Move ts aliases into defpreds, improve docstrings and comments commit 6ef710f Author: Adam Porter <[email protected]> Date: Thu Oct 3 10:11:17 2019 -0500 Tidy: Whitespace commit 1022c4b Author: Adam Porter <[email protected]> Date: Thu Oct 3 10:01:53 2019 -0500 WIP: (org-ql-search) Accept plain queries This way, helm-org-ql and org-ql-search accept queries in the same format. commit 1903e8f Author: Adam Porter <[email protected]> Date: Thu Oct 3 09:22:35 2019 -0500 WIP: Add more plain-query tests commit 9d58e33 Author: Adam Porter <[email protected]> Date: Thu Oct 3 09:17:07 2019 -0500 WIP: Return boolean from --plain-query, and only when needed commit 3770e18 Author: Adam Porter <[email protected]> Date: Thu Oct 3 08:59:55 2019 -0500 WIP: Parse numbers in key/value args e.g. "ts-a:on=-1" for yesterday, or "ts:on=0" or "ts:on=today" for today. commit 0f2b71b Author: Adam Porter <[email protected]> Date: Thu Oct 3 08:59:43 2019 -0500 WIP: Rename plain-query function commit be410a3 Author: Adam Porter <[email protected]> Date: Wed Oct 2 19:21:07 2019 -0500 WIP: Reverse query properly PEG returns queries backwards for some reason, so reverse them back to the order they were input in. commit c184091 Author: Adam Porter <[email protected]> Date: Wed Oct 2 19:17:20 2019 -0500 WIP: Fix previous commit Something weird happens when defining the function that's defined by the macro in the eval-when block: It seems that I have to eval it twice before it takes effect. So when I made the previous commit, the function defined at the time was working properly, but the function defined at the time was apparently not what was written in the file, but some intermediate state that had been written before, so I don't even know what the code was that was working properly. After some more fiddling, this seems to work properly. commit 4e7288d Author: Adam Porter <[email protected]> Date: Wed Oct 2 18:42:59 2019 -0500 WIP: Handle keyword/value queries commit f603d19 Author: Adam Porter <[email protected]> Date: Tue Oct 1 16:45:33 2019 -0500 WIP: Define pred aliases and input-query function with macros commit 217ef7d Author: Adam Porter <[email protected]> Date: Tue Oct 1 16:16:04 2019 -0500 WIP: Tidy: Don't quote the lambda I don't even know why I did that. commit 3130f65 Author: Adam Porter <[email protected]> Date: Fri Sep 27 23:20:50 2019 -0500 WIP: Parsing with peg Seems to be working pretty well. commit 0762825 Author: Adam Porter <[email protected]> Date: Tue Sep 10 16:17:50 2019 -0500 WIP: Rewriting Helm queries Some people are unaccustomed to parentheses, so we'll try to accommodate them.
1 parent cea8028 commit f4918b1

9 files changed

+1047
-521
lines changed

README.org

+44-147
Original file line numberDiff line numberDiff line change
@@ -6,13 +6,15 @@
66

77
[[https://melpa.org/#/org-ql][file:https://melpa.org/packages/org-ql-badge.svg]] [[https://stable.melpa.org/#/org-ql][file:https://stable.melpa.org/packages/org-ql-badge.svg]]
88

9-
~org-ql~ is a lispy query language for Org files. It allows you to find Org entries matching certain criteria and return a list of them or perform actions on them. Commands are also provided which display matching results.
9+
This package provides a query language for Org files. It offers two syntax styles: Lisp-like sexps and search engine-like keywords.
10+
11+
It includes three libraries: The =org-ql= library is flexible and may be used as a backend for other tools. The libraries =org-ql-search= and =helm-org-ql= provide interactive search commands and saved views.
1012

1113
* Contents
1214
:PROPERTIES:
1315
:TOC: this
1416
:END:
15-
- [[#examples][Examples]]
17+
- [[#screenshots][Screenshots]]
1618
- [[#installation][Installation]]
1719
- [[#usage][Usage]]
1820
- [[#commands][Commands]]
@@ -21,77 +23,15 @@
2123
- [[#changelog][Changelog]]
2224
- [[#notes][Notes]]
2325

24-
* Examples
26+
* Screenshots
2527

26-
More examples are available in [[examples.org]].
28+
[[images/org-ql-search.gif]]
2729

28-
#+BEGIN_SRC elisp
29-
;; Show entries that have any timestamp within the past week. Group
30-
;; by date using `org-super-agenda' with the `:auto-ts' group.
31-
(org-ql-search (org-agenda-files)
32-
'(ts :from -7 :to today)
33-
:title "Recent Items"
34-
:sort '(date priority todo)
35-
:groups '((:auto-ts t)))
36-
37-
;; Show a GTD-style "stuck projects" view: PROJECT tasks that have no
38-
;; descendants with the NEXT keyword. If you use a "project" tag
39-
;; instead of the to-do keyword, you could replace (todo "PROJECT")
40-
;; with (tags "project").
41-
(org-ql-search (org-agenda-files)
42-
'(and (todo "PROJECT")
43-
(not (descendants (todo "NEXT"))))
44-
:title "Stuck Projects")
45-
46-
;; Integrate `org-ql' into a custom Org Agenda command which inserts
47-
;; an `org-ql' block before the regular agenda:
48-
(setq org-agenda-custom-commands
49-
'(("ces" "Custom: Agenda and Emacs SOMEDAY [#A] items"
50-
((org-ql-block '(and (todo "SOMEDAY")
51-
(tags "Emacs")
52-
(priority "A")))
53-
(agenda)))))
30+
[[images/org-ql-search-snippet.png]]
5431

55-
;; Return a list of bills coming due, searching all Org Agenda files,
56-
;; sorted by deadline. The `auto' argument to `deadline' means to match
57-
;; entries whose deadlines fall within `org-deadline-warning-days'.
58-
;; `org-ql-query' works like `org-ql-select' but offers arguments named
59-
;; like SQL queries.
60-
(org-ql-query
61-
:select #'org-get-heading
62-
:from (org-agenda-files)
63-
:where '(and (not (done))
64-
(tags "bills")
65-
(deadline auto))
66-
:order-by 'deadline)
67-
;;=> ("TODO Electric bill" "TODO Water bill")
68-
69-
;; If you kept a database of music in an Org file, you could run a
70-
;; query like this to find tracks composed by Chopin that do not have
71-
;; their key recorded in the database. `org-ql-search' works like
72-
;; `org-ql-select' and displays results in an agenda-like buffer:
73-
(org-ql-search "~/org/music.org"
74-
'(and (property "genre" "classical")
75-
(property "composer" "Chopin")
76-
(not (property "key"))))
77-
78-
;; Set the tag "Emacs" on every entry in the inbox file that mentions
79-
;; "Emacs". `org-ql-select' works like `org-ql' but is a function
80-
;; rather than a macro. The bare-string query "Emacs" is equivalent
81-
;; to (regexp "Emacs").
82-
(org-ql-select "~/org/inbox.org"
83-
"Emacs"
84-
:action '(org-toggle-tag "Emacs" 'on))
85-
86-
;; Return a list of Org entry elements in the file "~/org/main.org"
87-
;; which have the SOMEDAY to-do keyword, are tagged "Emacs", and have
88-
;; priority B or higher.
89-
(org-ql "~/org/main.org"
90-
(and (todo "SOMEDAY")
91-
(tags "Emacs")
92-
(priority >= "B")))
93-
;;=> ((headline (:raw-value "org-board" :begin 1220270 :end 1220403 ...)) ...)
94-
#+END_SRC
32+
[[images/helm-org-ql.gif]]
33+
34+
[[images/org-ql-view-sidebar.gif]]
9535

9636
* Installation
9737
:PROPERTIES:
@@ -100,9 +40,9 @@ More examples are available in [[examples.org]].
10040

10141
The package may be installed directly from [[https://melpa.org/#/org-ql][MELPA]] or with other tools like [[https://framagit.org/steckerhalter/quelpa][Quelpa]].
10242

103-
After installation, you can use commands like ~org-ql-search~ immediately.
43+
After installation, you can use the commands without additional configuration. /Note: The command =helm-org-ql= only works if the package =helm-org= is installed; Helm is not a dependency of this package, so it's not automatically installed./
10444

105-
To use the functions and macros in your own Elisp code, load the libraries ~org-ql~ and/or ~org-ql-agenda~ with e.g. ~(require 'org-ql)~.
45+
To use the functions and macros in your own Elisp code, use libraries =org-ql= and =org-ql-view=.
10646

10747
** Quelpa
10848

@@ -134,7 +74,9 @@ These commands and functions are included:
13474
- ~org-ql-select~ (function)
13575
- ~org-ql-query~ (function)
13676

137-
Feedback on these APIs is welcome. Eventually, after being tested and polished, they will be considered stable.
77+
Feedback on these APIs is welcome. Eventually, after being tested and polished, they will be considered stable.
78+
79+
Lisp code examples are in [[examples.org]].
13880

13981
** Commands
14082
:PROPERTIES:
@@ -143,6 +85,8 @@ Feedback on these APIs is welcome. Eventually, after being tested and polished,
14385

14486
*** org-ql-search
14587

88+
/Note: This command supports both sexp queries and [[#non-sexp-query-syntax][non-sexp queries]]./
89+
14690
Read ~QUERY~ and search with ~org-ql~. Interactively, prompt for these variables:
14791

14892
~BUFFERS-FILES~: ~A~ list of buffers and/or files to search. Interactively, may also be:
@@ -163,22 +107,16 @@ Read ~QUERY~ and search with ~org-ql~. Interactively, prompt for these variable
163107
+ =g=: Refresh results.
164108
+ =C-x C-s=: Save query to variable ~org-ql-views~ (accessible with command ~org-ql-view~).
165109

166-
[[images/org-ql-search.gif]]
167-
168-
Here's an example of using it to generate an agenda-like view for certain files in a directory tree:
169-
170-
[[images/org-ql-search-snippet.png]]
110+
*Note:* The view buffer is currently put in ~org-agenda-mode~, which means that /some/ Org Agenda commands work, such as jumping to entries and changing item priorities (without necessarily updating the view). This feature is experimental and not guaranteed to work correctly with all commands. (It works to the extent it does because the appropriate text properties are placed on each item, imitating an Agenda buffer.)
171111

172112
*** helm-org-ql
173113

174-
This command displays matches with Helm. *Note:* Helm is not a package dependency, so this command only works if the package =helm-org= is installed.
114+
/Note: This command uses [[#non-sexp-query-syntax][non-sexp queries]]./
175115

176-
Note also that queries in this command are specially handled so that quotes around strings may be omitted for ease of typing.
116+
This command displays matches with Helm. *Note:* Helm is not a package dependency, so this command only works if the package =helm-org= is installed.
177117

178118
+ Press =C-x C-s= in the Helm session to save the results to an =org-ql-search= buffer.
179119

180-
[[images/helm-org-ql.gif]]
181-
182120
*** org-ql-view
183121

184122
Choose and display a view stored in ~org-ql-views~.
@@ -187,8 +125,6 @@ Choose and display a view stored in ~org-ql-views~.
187125

188126
Show a sidebar window listing views stored in =org-ql-views= for easy access. In the sidebar, press =RET= or =mouse-1= to show the view at point, and press =c= to customize the view at point.
189127

190-
[[images/org-ql-view-sidebar.gif]]
191-
192128
*** org-ql-view-recent-items
193129

194130
Show items in ~FILES~ from last ~DAYS~ days with timestamps of ~TYPE~. ~TYPE~ may be ~ts~, ~ts-active~, ~ts-inactive~, ~clocked~, ~closed~, ~deadline~, ~planning~, or ~scheduled~. =FILES= defaults to those returned by the function =org-agenda-files=.
@@ -201,11 +137,32 @@ Show a sparse tree for ~QUERY~ in ~BUFFER~ and return number of results. The tr
201137

202138
** Queries
203139

204-
A query is a lisp form which may contain arbitrary lisp forms, as well as certain built-in predicates. It is byte-compiled into a predicate function which is tested with point on each heading in an Org buffer; when it returns non-nil, the heading matches the query.
140+
An =org-ql= query is a lisp form which may contain arbitrary lisp forms, as well as certain built-in predicates. It is byte-compiled into a predicate function which is tested with point on each heading in an Org buffer; when it returns non-nil, the heading matches the query.
205141

206142
*Notes:*
207143
+ Bare strings like ~"string"~ are automatically converted to ~(regexp "string")~ predicates.
208-
+ Standard numeric comparator function symbols (~<~, ~<=~, ~>~, ~>=~, ~=~ ) need not be quoted when passed as an argument to these predicates. The resemblance to infix notation is coincidental. See examples in documentation.
144+
+ Standard numeric comparator function symbols (~<~, ~<=~, ~>~, ~>=~, ~=~ ) need not be quoted when passed as an argument to predicates which accept them. The resemblance to infix notation is coincidental.
145+
146+
*** Non-sexp query syntax
147+
:PROPERTIES:
148+
:TOC: ignore
149+
:END:
150+
151+
The command =org-ql-search= also accepts, and the command =helm-org-ql= only accepts, an alternative, non-sexp query syntax. The syntax is simple, and a few examples of queries in both syntaxes should suffice. By default, when multiple predicates are used, they are combined with boolean =and=.
152+
153+
| Sexp syntax | Non-sexp syntax |
154+
|-------------------------------------------------+-----------------------------------------|
155+
| ~(todo)~ | ~todo:~ |
156+
| ~(todo "SOMEDAY")~ | ~todo:SOMEDAY~ |
157+
| ~(todo "SOMEDAY" "WAITING")~ | ~todo:SOMEDAY,WAITING~ |
158+
| ~(ts :on today)~ | ~ts:on=today~ |
159+
| ~(ts-active :from "2017-01-01" :to "2018-01-01")~ | ~ts-active:from=2017-01-01,to=2018-01-01~ |
160+
| ~(clocked :on -1)~ | ~clocked:on=-1~ |
161+
| ~(heading "quoted phrase" "word")~ | ~heading:"quoted phrase",word~ |
162+
| ~(and (tags "book" "books") (priority "A"))~ | ~tags:book,books priority:A~ |
163+
| ~(priority >= B)~ | ~priority:A,B~ |
164+
165+
Note that the =priority= predicate does not support comparators in the non-sexp syntax, so multiple priorities should be passed instead, as seen in the last example.
209166

210167
*** Predicates
211168
:PROPERTIES:
@@ -297,67 +254,6 @@ However, the ~org-ql-block~ version runs in about 1/5th the time.
297254

298255
The variable =org-ql-block-header= may be bound to a string to use as the block header, otherwise the header is formed automatically.
299256

300-
**** Macro: ~org-ql-agenda~
301-
302-
This macro is like ~org-ql~, but it presents matching entries in an Agenda-like view. It's compatible with [[https://github.com/alphapapa/org-super-agenda][org-super-agenda]], which provides grouping. For example:
303-
304-
#+BEGIN_SRC elisp
305-
(org-ql-agenda "~/src/emacs/org-super-agenda/test/test.org"
306-
(and (or (ts-active :on today)
307-
(deadline auto)
308-
(scheduled :to today))
309-
(not (done)))
310-
:title "My Agenda View"
311-
;; The `org-super-agenda-groups' setting is used automatically when set, or it
312-
;; may be overriden by specifying it here:
313-
:super-groups ((:name "Bills"
314-
:tag "bills")
315-
(:todo ("SOMEDAY" "TO-READ" "CHECK" "TO-WATCH" "WATCHING")
316-
:order 7)
317-
(:name "Personal"
318-
:habit t
319-
:tag "personal"
320-
:order 3)
321-
(:todo "WAITING"
322-
:order 6)
323-
(:priority "A" :order 1)
324-
(:priority "B" :order 2)
325-
(:priority "C" :order 2)))
326-
#+END_SRC
327-
328-
Which presents this buffer:
329-
330-
[[images/screenshot.png]]
331-
332-
*Note:* The view buffer is currently put in ~org-agenda-mode~, which means that /some/ Org Agenda commands work, such as jumping to entries and changing item priorities (without necessarily updating the view). This feature is experimental and not guaranteed to work correctly with all commands. (It works to the extent it does because the appropriate text properties are placed on each item, imitating an Agenda buffer.)
333-
334-
Here are some other examples:
335-
336-
#+BEGIN_SRC elisp
337-
;; Show an agenda-like view of items in "~/org/main.org" with TODO and
338-
;; SOMEDAY keywords which are tagged "computer" or "Emacs" and in the
339-
;; category "main":
340-
(org-ql-agenda "~/org/main.org"
341-
(and (todo "TODO" "SOMEDAY")
342-
(tags "computer" "Emacs")
343-
(category "main")))
344-
345-
;; Show an agenda-like view of all habits in all agenda files:
346-
(org-ql-agenda
347-
(habit))
348-
349-
;; Show an agenda-like view similar to a "traditional" Org Agenda with
350-
;; Log Mode turned on.
351-
(org-ql-agenda
352-
(or (and (not (done))
353-
(or (habit)
354-
(deadline auto)
355-
(scheduled :to today)
356-
(ts-active :on today)))
357-
(closed :on today))
358-
:sort (date priority todo))
359-
#+END_SRC
360-
361257
*** Listing / acting-on results
362258

363259
**** Function: ~org-ql-select~
@@ -470,6 +366,7 @@ Expands into a call to ~org-ql-select~ with the same arguments. For convenience
470366
** 0.3-pre
471367

472368
*Added*
369+
+ Alternative, non-sexp query syntax for commands =org-ql-search= and =helm-org-ql=. See [[#non-sexp-query-syntax][documentation]].
473370
+ Command =helm-org-ql=.
474371
+ Command =org-ql-sparse-tree=, like =org-sparse-tree= for =org-ql= queries. (Thanks to [[https://github.com/akirak][Akira Komamura]].)
475372
+ Command =org-ql-view-sidebar=.

0 commit comments

Comments
 (0)