Skip to content

Documentation of ⎕NINFO callbacks #118

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 9 commits into from
May 22, 2025
120 changes: 11 additions & 109 deletions language-reference-guide/docs/system-functions/ncopy.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,15 +25,15 @@ Source and destination path names may be full or relative (to the current workin



If `X` specifies an existent directory then each source in `Y` is copied into that directory, otherwise `X` specifies the name of the copy. `X` must specify an existent directory if the source contains multiple names or if the Wildcard option is set.
If `X` specifies an existent directory then each source in `Y` is copied into that directory, otherwise `X` specifies the name of the copy. `X` must specify an existent directory if the source contains multiple names or if the **Wildcard** option is set.


The shy result `R` contains count(s) of top-level items copied. If `Y` is a single source name, `R` is a scalar otherwise it is a vector of the same length as `Y`.

## Variant Options


`⎕NCOPY` may be applied using the  Variant operator with the options Wildcard (the Principal option), IfExists, PreserveAttributes and ProcessCallback.
`⎕NCOPY` may be applied using the _variant_ operator with the options **Wildcard** (the Principal option), **IfExists**, **PreserveAttributes** and **ProgressCallback**.

## Wildcard Option (Boolean)

Expand All @@ -43,13 +43,13 @@ The shy result `R` contains count(s) of top-level items copied. If `Y` is a sing
|`1`|The name or names in `Y` that specify the *base name* and *extension* (see [NParts](./nparts.md) ), may also contain the wildcard characters "?" and "*". An asterisk is a substitute for any 0 or more characters in a file name or extension; a question-mark is a substitute for any single character.|


Note that when Wildcard is 1, element(s) of `R` can be 0, 1 or `>1`. If Wildcard is 0, elements of `R` are always 1.
Note that when **Wildcard** is 1, element(s) of `R` can be 0, 1 or `>1`. If **Wildcard** is 0, elements of `R` are always 1.


## IfExists Option


The IfExists variant option determines what happens when a source file is to be copied to a target file that already exists. It does not apply to directories, only to the files within them.
The **IfExists** variant option determines what happens when a source file is to be copied to a target file that already exists. It does not apply to directories, only to the files within them.


|Value |Description |
Expand All @@ -61,15 +61,15 @@ The IfExists variant option determines what happens when a source file is to be



The following cases cause an error to be signalled regardless of the value of the IfExists variant.
The following cases cause an error to be signalled regardless of the value of the **IfExists** variant.

- If the source specifies a directory and the destination specifies an existing file.
- If the source specifies a file and the same base name exists as a sub-directory in the destination.

## PreserveAttributes Option (Boolean)


The PreserveAttributes variant option determines whether or not file attributes are preserved. It does not apply to directories, only to files.
The **PreserveAttributes** variant option determines whether or not file attributes are preserved. It does not apply to directories, only to files.


|---|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
Expand Down Expand Up @@ -177,113 +177,15 @@ backups/def_uk.dse
backups/UserCommand20.cache
```

## ProgressCallback Option

The **ProgressCallback** variant option is described in the [Dyalog Programming Reference Guide](../../../programming-reference-guide/native-files#progress-callbacks). following is specific to `⎕NCOPY`:

* The first element of the right argument to the callback function is the character vector `'⎕NCOPY'`.

## Notes

- The special directories `.` and `..` can never be copied into an existing directory.
- If any source name is a symbolic link it is dereferenced; that is, the source or directory it references is copied rather than the link itself.
- In the result `R`, a directory together with all its contents is counted once. A directory may be partially copied if the IfExists option is set to `'Replace'` or `'ReplaceIfNewer'`).
- In the result `R`, a directory together with all its contents is counted once. A directory may be partially copied if the **IfExists** option is set to `'Replace'` or `'ReplaceIfNewer'`).
- If an error occurs during the copy process then processing will immediately stop and an error will be signalled. The operation is not atomic; some items may be copied before this happens. In the event of an error there will be no result and therefore no indication of how many names were copied before the error occurred.

### ProgressCallback Option

### Overview


If this option is enabled, the system function invokes an APL callback function as the file operation (move or copy) proceeds. A system object is used to communicate between the system function and the callback. The file operation has 4 distinct stages:

1. The start of the operation. The callback is invoked before any files are scanned or processed. This gives the application the opportunity to set parameters that control the frequency of callbacks during the operation itself.
2. The optional scan phase during which the system function enumerates the files that will be involved in the copy or move operation. The file count obtained is used to set the `Limit` field. The application may use this subsequently to indicate the degree of progress.
3. The main processing (move or copy) of the files.
4. The end of the operation.


The callback function is invoked once at the start of the operation, during the (optional) scan and processing stages, and finally once at the end of the operation. During the scan and processing stages, the `Skip` and `Delay` options provide alternative ways to control the frequency with which the callback is invoked.


If both options are 0, the callback will be invoked after every file is processed. However, if there are a large number of small files involved, and you simply want to update a progress bar, this may prove to be unnecessarily frequent, and will increase the total time required to complete the operation.


If you want to update a progress bar regularly (for example every second), the `Delay` option (1000 = 1 second) is the better choice. In other circumstances, you might choose to use `Skip`.


If you use both options, the callback will be invoked when *both* apply, so if you set `Skip` to 10 and `Delay` to 5000, the callback will be invoked after at least 10 files have been processed and at least 5 seconds have elapsed since the previous invocation of the callback.


The value of the ProgressCallback variant option may be:


|---------|------------------------------------------------------------------------------------------------------------------------|
|`fn` |The name of the callback function. |
|`fn data`|The name of the callback function, and an array or namespace which is to be passed to the callback in its left argument.|



The right argument given to the callback function is a 3-element vector:


|-----|--------|------------------------------------------------------------------------------------------------------------------------|
|`[1]`|Function|Character vector which identifies the function that caused the callback to be executed; either `'⎕NCOPY'` or `'⎕NMOVE'.`|
|`[2]`|Event |Character vector describing the event that lead to the callback being executed. See below. |
|`[3]`|Info |Reference to a namespace containing information about the event. See below. |


### Event



Event is a character vector which indicates the stage of the copy or move operation..


|---|---|
|`'Start'`|Reported by the first invocation of the callback which occurs before any files are scanned or processed. This may be used to set the parameters that control the operation. See **Options** .|
|`'Scan'`|Indicates that the system function is in the initial phase of scanning the files in order to calculate `Limit` . See **ScanFirst** .|
|`'Progress'`|Indicates that the system function is at the main stage of the operation and is moving or copying the files.|
|`'Done'`|Indicates that all files have been processed.|



Note that there will always be at least 2 invocations of the callback, to indicate the start and end of the operation.


### Info


Info is a ref to a namespace that contains information about the event. This namespace persists for the duration of the execution of the system function and contains the following fields:


|---|---|
|`Progress`|A number between `0` and `Limit` . When the event code is `'Start'` , `Progress` is `0` . Every time a file or directory is processed, `Progress` is increased by 1. Finally when the event code is `'Done'` , `Progress` will be equal to `Limit` .|
|`Limit`|The maximum value of `Progress` . This value might change during the file operation if it doesn't do a full discovery first (the `ScanFirst` option is 0), or if the file structure changes between the scan and the copy/move.|
|`Last`|A vector of file names which have been processed since the last invocation of the callback function. The user can specify the maximum length of this vector by setting the `LastFileCount` option. The names in this list are the source names, and not the destination names. The `Last` vector is always empty when the event is `'Start'` , and it is cleared when going from the `'Scan'` phase to the `'Progress'` phase, to avoid any confusion.|
|`Data`|A field that is reserved for the user to store data which persists between invocations of the callback. It could for example be used to keep a sequence number, to count the number of times the callback had been run.|
|`Options`|This is a namespace which contains the information that controls the future execution of the callback. The options persist between the calls to the callback, so there is no need to set them again unless they should be changed. The fields and their default values are described below.|



### Options


This is a namespace which contains options that control future invocations of the callback. The options persist between these invocations, so there is no need to set them again unless they should be changed. The fields and their default values are:


|Field|Default|Description|
|---|---|---|
|`ScanFirst`|1|Specifies if the file operation should do a "scan pass" before moving/copying the files. This stage just enumerates the files to determine how many there are. This will ensure `Limit` has a realistic value when the actual processing of the files happens. The overhead is small in comparision with the time it takes to process the files. The `ScanFirst` field is only inspected right after the first invocation of the callback function, with the event code `'Start'` .|
|`Delay`|0|Specifies the number of milliseconds to wait, until the callback will be called again. If all file operations finish before this time, the callback function is called anyway, with the event code `'Done'` . If a slow file operation is happening (such as copying a big file), the actual delay before the callback is invoked might be longer than the value of `Delay` .|
|`Skip`|0|Specifies a number of files to skip between invocations of the callback function. If you are only interested in getting a callback for each 10th file, you should set this option to 9 for example.|
|`LastFileCount`|1|An integer, specifying the maximum number of the latest filenames to be stored in the `Last` field. The default is to only store the last file processed, but if `Delay` or `Skip` are non-zero, multiple files could have been processed between calls to the callback function. A value of 5 for example, will make sure that the 5 last files processed before calling the callback, will have their names in the `Last` field. The `Last` field might have fewer elements than `LastFileCount` , if the number of files processed since the last call is less than `LastFileCount` . The special value `¯1` indicates that the `Last` field should contain **all** the last files since the last call (no limit).|



The result of the callback function must be a Boolean scalar, indicating whether or not the `⎕NCOPY` or `⎕NMOVE` should continue or stop.


1: Execution should continue.


0: Execution should stop. In this case, an `INTERRUPT` (event 1003) is signalled.



19 changes: 13 additions & 6 deletions language-reference-guide/docs/system-functions/ninfo.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ This function queries or sets information about one or more files or directories

## Variant Options

`⎕NINFO` may be applied using the Variant operator with the options Wildcard (the Principal option), Recurse and Follow.
`⎕NINFO` may be applied using the _variant_ operator with the options **Wildcard** (the Principal option), **Recurse**, **Follow** and **ProgressCallback**.

## Wildcard Option (Boolean)

Expand All @@ -25,9 +25,9 @@ This function queries or sets information about one or more files or directories

|---|---|
|0 { .shaded } |the name(s) in `Y` are searched for only in the corresponding specified directory.|
|`1`|the name(s) in `Y` are searched for in the corresponding specified directory as well as all sub-directories. If Wildcard is also 1, the wild card search is performed recursively.|
|`1`|the name(s) in `Y` are searched for in the corresponding specified directory as well as all sub-directories. If **Wildcard** is also 1, the wild card search is performed recursively.|
|`1 n`|the name(s) in `Y` are searched for in the corresponding specified directory as well as its sub-directories to the n <sup>th</sup> -level sub-directory. If n is 0, no sub-directories are searched. If n is `¯1` all sub-directories are searched.|
|`2 (n)`|same as 1 but if any unreadable directories are encountered they are skipped (whereas if Recurse is `1 (n)` , `⎕NINFO` stops and generates an error).|
|`2 (n)`|same as 1 but if any unreadable directories are encountered they are skipped (whereas if **Recurse** is `1 (n)` , `⎕NINFO` stops and generates an error).|

## Follow Option (Boolean)

Expand Down Expand Up @@ -80,11 +80,18 @@ If the Wildcard option is not enabled (the default) then `Y` specifies exactly o

If the Wildcard option is enabled, zero or more files and/or directories may match the pattern in `Y`. In this case each element in `R` is a vector of property values for each of the files. Note that no error will be signalled if no files match the pattern.

When using the Wildcard option, matching of names is done case insensitively on Windows and macOS, and case sensitively on other platforms. The names '.' and '..' are excluded from any matches. The order in which the names match is not defined.
When using the **Wildcard** option, matching of names is done case insensitively on Windows and macOS, and case sensitively on other platforms. The names '.' and '..' are excluded from any matches. The order in which the names match is not defined.

## ProgressCallback Option

The **ProgressCallback** variant option is described in the [Dyalog Programming Reference Guide](../../../programming-reference-guide/native-files#progress-callbacks). The following is specific to `⎕NINFO`:

* The first element of the right argument to the callback function is the character vector `'⎕NINFO'`.
* The third element of the right argument (the information namespace) contains an extra field named `Info`, which is a vector with the same length as the `Last` field. Each element of the `Info` vector contains the information requested by the `⎕NINFO` call for the corresponding filename in `Last`.

## Note

On non-Windows platforms, file names are exposed by the Operating System using UTF-8 encoding which Dyalog translates internally to characters.
On platforms other than Microsoft Windows, file names are exposed by the operating system using UTF-8 encoding, which Dyalog translates internally to characters.

In the Unicode Edition, if the UTF-8 encoding is invalid, Dyalog replaces each offending byte with a unique Unicode symbol (in the *Low Surrogate Area* of the Unicode charts) that is mapped back to the original byte by the other system functions (including `⎕NTIE` and `⎕NDELETE`) that take native file names as arguments. The display of a file name containing these mapped bytes may appear strange.

Expand Down Expand Up @@ -163,7 +170,7 @@ C:/Users/Pete/Documents/Dyalog APL-64 16.0 Unicode Files/
└──────────────────────┘
```

The next set of examples, illustrate the use of the Recurse variant option to limit the sub-directory depth.
The next set of examples, illustrate the use of the **Recurse** variant option to limit the sub-directory depth.
```apl
Y←'d:\bouzouki\*.*'
⍴⊃0(⎕NINFO⍠('Wildcard' 1)('Recurse' 0))Y
Expand Down
Loading