Skip to content

Commit 217b3a6

Browse files
authored
Add SQL Scheduler docs (#59)
* Add SQL Scheduler docs
1 parent e1782de commit 217b3a6

5 files changed

+122
-0
lines changed
Loading
Loading
Loading
Loading

docs/reference/overview.md

Lines changed: 122 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -38,6 +38,7 @@ more information on CrateDB Cloud-related terminology.
3838
- [Backups](#overview-cluster-backups)
3939
- [Cluster Cloning](#overview-cluster-cloning)
4040
- [Failed cloning](#overview-cluster-cloning-fail)
41+
- [SQL Scheduler](#overview-sql-scheduler)
4142
- [Scale](#overview-cluster-settings-scale)
4243
- [Manage](#overview-cluster-manage)
4344
- [Community](#overview-community)
@@ -603,6 +604,127 @@ screen.
603604

604605
![Cloud Console cluster failed cloning](../_assets/img/cluster-clone-failed.png)
605606

607+
(overview-sql-scheduler)=
608+
:::
609+
## SQL Scheduler
610+
:::
611+
612+
The SQL Scheduler is designed to automate routine database tasks by scheduling
613+
SQL queries to run at specific times, in UTC time. This feature
614+
supports creating job descriptions with valid
615+
[cron patterns](https://www.ibm.com/docs/en/db2oc?topic=task-unix-cron-format)
616+
and SQL statements, enabling a wide range of tasks. Users can manage these jobs
617+
through the Cloud UI, adding, removing, editing, activating, and deactivating
618+
them as needed.
619+
620+
### Use Cases
621+
622+
- Deleting old/redundant data to maintain database efficiency.
623+
- Regularly updating or aggregating table data.
624+
- Automating export and import of data.
625+
626+
:::{note}
627+
- The SQL Scheduler is automatically available for all newly deployed clusters.
628+
- For existing clusters, the feature can be enabled on demand.
629+
(Contact support for activation.)
630+
:::
631+
632+
### Accessing and Using the SQL Scheduler
633+
634+
SQL Scheduler can be found in "SQL Scheduler" tab in the left-hand navigation
635+
menu. There are 2 tabs on the SQL Scheduler page:
636+
637+
:::{tab} Overview
638+
<br>
639+
640+
**Overview** shows a list of your existing jobs. In the list you can
641+
activate/deactivate each job with a toggle in the "Active" column. You can
642+
also edit and delete jobs with buttons on the right side of the list.
643+
644+
![SQL Scheduler overview](../_assets/img/cluster-sql-scheduler-overview.png)
645+
:::
646+
647+
:::{tab} Logs
648+
<br>
649+
650+
**Logs** shows a list of *scheduled* job runs, whether they failed or
651+
succeeded, execution time, run time, and the error in case they were
652+
unsuccessful. In case of an error, more details can be viewed showing the
653+
executed query and a stack trace. You can filter the logs by status, or by
654+
specific job.
655+
656+
![SQL Scheduler overview](../_assets/img/cluster-sql-scheduler-logs.png)
657+
:::
658+
659+
### Examples
660+
661+
::::{tab} Cleanup of old files
662+
<br>
663+
664+
Cleanup tasks represent a common use case for these types of automated jobs.
665+
This example deletes records older than 30 days, from a specified table once a
666+
day:
667+
668+
:::{code} sql
669+
DELETE FROM "sample_data"
670+
WHERE
671+
"timestamp_column" < NOW() - INTERVAL '30 days';
672+
:::
673+
674+
How often you run it of course depends on you, but once a day is common for
675+
clean up. This expression runs every day at 2:30 PM UTC:
676+
677+
Schedule: `30 14 * * *`
678+
679+
![SQL Scheduler overview](../_assets/img/cluster-sql-scheduler-example-cleanup.png)
680+
::::
681+
682+
::::{tab} Copying logs into persistent table
683+
<br>
684+
685+
Another useful example might be copying data to another table for archival
686+
purposes. This specifically copies from system logs table into one of our
687+
own tables.
688+
689+
:::{code} sql
690+
CREATE TABLE IF NOT EXISTS "logs"."persistent_jobs_log" (
691+
"classification" OBJECT (DYNAMIC),
692+
"ended" TIMESTAMP WITH TIME ZONE,
693+
"error" TEXT,
694+
"id" TEXT,
695+
"node" OBJECT (DYNAMIC),
696+
"started" TIMESTAMP WITH TIME ZONE,
697+
"stmt" TEXT,
698+
"username" TEXT,
699+
PRIMARY KEY (id)
700+
) CLUSTERED INTO 1 SHARDS;
701+
702+
INSERT INTO
703+
"logs"."persistent_jobs_log"
704+
SELECT
705+
*
706+
FROM
707+
sys.jobs_log
708+
ON CONFLICT ("id") DO NOTHING;
709+
:::
710+
711+
In this example, we schedule the job to run every hour:
712+
713+
Schedule: `0 * * * *`
714+
715+
716+
![SQL Scheduler overview](../_assets/img/cluster-sql-scheduler-example-copying.png)
717+
::::
718+
719+
:::{note}
720+
Limitations and Known Issues:
721+
722+
* Only one job can run at a time; subsequent jobs will be queued until the
723+
current one completes.
724+
* Long-running jobs may block the execution of queued jobs, leading to
725+
potential delays.
726+
:::
727+
606728
(overview-cluster-settings-scale)=
607729
### Scale
608730

0 commit comments

Comments
 (0)