1.6.0

Highlights

• Added proxy support (#193)
• Added support for inferring NULL values passed as query parameters (#189)
• Fixed bug with NULL handling for Arrow results (#195)

Full diff: 1.5.0...1.6.0

Proxy support

This feature allows to pass through proxy all the requests library makes. By default, proxy is disabled.
To enable proxy, pass a configuration object to DBSQLClient.connect:

client.connect({
    // pass host, path, auth options as usual
    proxy: {
      protocol: 'http',  // supported protocols: 'http', 'https', 'socks', 'socks4', 'socks4a', 'socks5', 'socks5h'
      host: 'localhost', // proxy host (string)
      port: 8070,        // proxy port (number)
      auth: {            // optional proxy basic auth config
        username: ...
        password: ...
      },
    },
  })

Note: using proxy settings from environment variables is currently not supported

1.5.0

Highlights

• Added OAuth M2M support (#168, #177)
• Added named query parameters support (#162, #175)
• runAsync options is now deprecated (#176)
• Added staging ingestion support (#164)

Full diff: 1.4.0...1.5.0

Databricks OAuth support

Databricks OAuth support added in v1.4.0 is now extended with M2M flow. To use OAuth instead of PAT, pass
a corresponding auth provider type and options to DBSQLClient.connect:

// instantiate DBSQLClient as usual

client.connect({
  // provide other mandatory options as usual - e.g. host, path, etc.
  authType: 'databricks-oauth',
  oauthClientId: '...', // optional - overwrite default OAuth client ID
  azureTenantId: '...', // optional - provide custom Azure tenant ID
  persistence: ...,     // optional - user-provided storage for OAuth tokens, should implement OAuthPersistence interface
})

U2M flow involves user interaction - the library will open a browser tab asking user to log in. To use this flow,
no other options are required except for authType.

M2M flow does not require any user interaction, and therefore is a good option, say, for scripting. To use this
flow, two extra options are required for DBSQLClient.connect: oauthClientId and oauthClientSecret.

Also see Databricks docs
for more details about Databricks OAuth.

Named query parameters

v1.5.0 adds a support for query parameters.
Currently only named parameters are supported.

Basic usage example:

// obtain session object as usual

const operation = session.executeStatement('SELECT :p1 AS "str_param", :p2 AS "number_param"', {
  namedParameters: {
    p1: 'Hello, World',
    p2: 3.14,
  },
});

The library will infer parameter types from passed primitive objects. Supported data types include booleans, various
numeric types (including native BigInt and Int64 from node-int64), native Date type, and string.

It's also possible to explicitly specify the parameter type by passing DBSQLParameter instances instead of primitive
values. It also allows one to use values that don't have a corresponding primitive representation:

import { ..., DBSQLParameter, DBSQLParameterType } from '@databricks/sql';

// obtain session object as usual

const operation = session.executeStatement('SELECT :p1 AS "date_param", :p2 AS "interval_type"', {
  namedParameters: {
    p1: new DBSQLParameter({
      value: new Date('2023-09-06T03:14:27.843Z'),
      type: DBSQLParameterType.DATE, // by default, Date objects are inferred as TIMESTAMP, this allows to override the type
    }),
    p2: new DBSQLParameter({
      value: 5, // INTERVAL '5' DAY
      type: DBSQLParameterType.INTERVALDAY
    }),
  },
});

Of course, you can mix primitive values and DBSQLParameter instances.

runAsync deprecation

Starting with this release, the library will execute all queries asynchronously, so we have deprecated
the runAsync option. It will be completely removed in v2. So you should not use it going forward and remove all
the usages from your code before version 2 is released. From user's perspective the library behaviour won't change.

Data ingestion support

This feature allows you to upload, retrieve, and remove unity catalog volume files using SQL PUT, GET and REMOVE commands.

1.4.0

• Added Cloud Fetch support (#158)
• Improved handling of closed sessions and operations (#129).
  Now, when session gets closed, all operations associated with it are immediately closed.
  Similarly, if client gets closed - all associated sessions (and their operations) are closed as well.

Full diff: 1.3.0...1.4.0

Notes:

Cloud Fetch is disabled by default. To use it, pass useCloudFetch: true to IDBSQLSession.executeStatement(). For example:

// obtain session object as usual
const operation = session.executeStatement(query, {
  runAsync: true,
  useCloudFetch: true,
});

Note that Cloud Fetch is effectively enabled only for really large datasets, so if the query returns only few thousands records, Cloud Fetch won't be enabled no matter what useCloudFetch setting is. Also gentle reminder that for large datasets it's better to use fetchChunk instead of fetchAll to avoid OOM errors:

do {
  const chunk = await operation.fetchChunk({ maxRows: 100000 });
  // process chunk here
} while (await operation.hasMoreRows());

1.3.0

• Implemented automatic retry for some HTTP errors (429, 503) (#127)
• Implemented request timeout + added option to configure it (#148)
• Added OAuth (U2M) support for AWS and Azure (#147 and #154)
• Fixed bug: for Arrow results, null values were ignored (@ivan-parada #151)

1.2.1

• Added Azure AD support (#126)
• Improved direct results handling (#134)
• Updated API endpoint references in docs and samples (#137)
• Code refactoring to improve maintainability

1.2.0

What's Changed

• Added Apache Arrow support (#94)
• Auth provider is now configurable (#120)

1.1.1

Fix: patch needed for improved error handling wasn't applied when installing v1.1.0

1.1.0

What's Changed

• Fix: now library will not attempt to parse column names and will use ones provided by server
  (#84)
• Better error handling: more errors can now be handled in specific .catch() handlers instead of being
  emitted as a generic error event (#99)
• Fixed error logging bug (attempt to serialize circular structures) (#89)
• Fixed some minor bugs and regressions

Full Changelog: 1.0.0...1.1.0

Upgrading

No specific actions required. Revisit error handling code if you used an error event - now most errors will be handled in specific .catch() handlers

1.0.0

What's Changed

• [PECO-237] Direct results support for all operations by @kravets-levko in #63
• [PECO-217] DBSQLClient.openSession refactoring by @kravets-levko in #64
• Fixed bug with spread operator crashing memory if chunk is too big by @nithinkdb in #66
• [PECO-217] Expose runAsync option for operations that support it by @kravets-levko in #65
• Enforce minimum TLS version by @kravets-levko in #69
• Remove deprecated option confOverlay by @kravets-levko in #68
• Make IOperation.getSchema wait for operation to complete by @kravets-levko in #67
• Add TSDoc comments for all public APIs. by @nithinkdb in #71
• Add client side logging by @nithinkdb in #72
• Fix e2e tests by @kravets-levko in #79
• [PECO-348] Enable direct results by default by @kravets-levko in #70
• Removed old APIs by @nithinkdb in #81
• Updated version by @nithinkdb in #80
• Fix mistake in test by @kravets-levko in #82
• Fixed bug in Logger, cleaned up types by @nithinkdb in #83

Full Changelog: 0.1.8-beta.2...1.0.0

Upgrading

All statements now execute with direct results by default. To turn off direct results, set the maxRows option to null in ExecuteStatementOptions.
The Node.js Connector is officially released!

0.1.8-beta.2

• Operations will wait for cluster to start instead of failing
• Added support for DirectResults, which speeds up data fetches by reducing the number of server roundtrips when possible
• DBSQLOperation interface simplified: HiveUtils were removed and replaced with new methods
  DBSQLOperation.fetchChunk/DBSQLOperation.fetchAll. New API implements all necessary waiting
  and data conversion routines internally
• Better TypeScript support
• Thrift definitions updated to support additional Databricks features
• User-agent string updated; a part of user-agent string is configurable through DBSQLClient's clientId option
• Connection now uses keep-alive (not configurable at this moment)
• DBSQLClient now prepends slash to path when needed
• DBSQLOperation: default chunk size for data fetching increased from 100 to 100.000

Upgrading

DBSQLClient.utils was permanently removed. Code which used utils.waitUntilReady, utils.fetchAll
and utils.getResult to get data should now be replaced with the single DBSQLOperation.fetchAll method.
Progress reporting, previously supported by utils.waitUntilReady, is now configurable via
DBSQLOperation.fetchChunk/DBSQLOperation.fetchAll options. DBSQLOperation.setMaxRows also became
an option of methods mentioned above.