Skip to content

Commit ef0f105

Browse files
authored
Further fixes and refactors for 1.4.0 (#63)
* headers integration testing * better encapsulation on the core classes * updated non-astra example to use updateDbNamespace * reinstated example deps to npm versions * overhauled how tests run * added some missing typescript lang specifiers in tsdoc examples * throw error on missing namespace * more work on test overhaul * tiny bit of tweaking to the test script * commented test script a bit * version script + increase a tests timeout * a * change default test timeout to 1m * use export type where possible * tiny tweak to build script * update to build pipeline to delete "empty" files * fixed bug with datapidbadmin * fixed a couple documentation errors * reverted example deps to npm versions * added exit to if invalid flag passed to test script * better help/error msg for test script * updated vectorize tests to work with non-astra * updated vectorize tests to work with bedrock * linting fix * added missing export
1 parent 5f4f8a2 commit ef0f105

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

66 files changed

+455
-162
lines changed

.np-config.json

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,3 @@
11
{
2-
"testScript": "test:prerelease"
2+
"testScript": "test -- --prerelease"
33
}

DEVGUIDE.md

Lines changed: 21 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -10,6 +10,12 @@ Prerequisites:
1010
- A clean AstraDB instance with two keyspaces—`default_keyspace` and `other_keyspace`
1111
- Copy the `.env.example` file and create a new `.env` file following the example template
1212

13+
```shell
14+
npm run test -- [--all | --light | --coverage | --prerelease] [-f <filter>] [-b] [--args <raw_args>]
15+
# or
16+
npm run test -- <--types>
17+
```
18+
1319
```shell
1420
# Run both unit and integration tests
1521
npm run test
@@ -20,10 +26,21 @@ npm run test -- -f 'unit.'
2026
# Run only integration tests
2127
npm run test -- -f 'integration.'
2228

29+
# Run all possible tests
30+
npm run test -- --all
31+
32+
# Run all possible integration tests
33+
npm run test -- --all -f 'integration.'
34+
35+
# Run all tests that aren't admin/long/vectorize
36+
npm run test -- --light -f 'integration.'
37+
2338
# Run tsc with the noEmit flag to check for type errors
24-
npm run test:types
39+
npm run test -- --types
2540
```
2641

42+
(bun does not need the extra initial `--` like npm does)
43+
2744
### Running the tests on local Stargate
2845
You can do `sh scripts/start-stargate-4-tests.sh` to spin up an ephemeral Data API on DSE instance which automatically
2946
creates the required keyspaces and destroys itself on exit.
@@ -56,11 +73,6 @@ env ASTRA_RUN_LONG_TESTS=1 npm run test
5673

5774
The `PROD` and `DEV` tags are enabled/disabled automatically, inferred from the astra endpoint URL.
5875

59-
Use the following to run tests with ADMIN and LONG tags automatically enabled (note that doesn't include vectorize tests):
60-
```shell
61-
npm run test:all
62-
```
63-
6476
### Adding your own tagged tests
6577
To enforce the tags, use the `assertTestsEnabled` function from `test/fixtures.ts`, which will skip the function if the
6678
given tag is not enabled.
@@ -122,10 +134,10 @@ See `vectorize_credentials.example.json` for—guess what—an example.
122134

123135
To run coverage testing, run the following command:
124136
```shell
125-
npm run test:coverage
137+
npm run test -- --coverage
126138
```
127139

128-
This uses `test:all` under the hood, as well as a "bail early" flag as there's not really a point continuing to run
140+
This uses `test --all` under the hood, as well as a "bail early" flag as there's not really a point continuing to run
129141
tests if one of them fails, as the coverage report will be impacted.
130142

131143
## Linting
@@ -143,5 +155,5 @@ To build it, just run `npm run build`, which does the following:
143155
- Deletes any extraneous `.d.ts` files
144156

145157
## Publishing
146-
I heavily recommend using [np](https://github.com/sindresorhus/np) to publish the package. Running it will involve running `test:prerelease`, and the
158+
I heavily recommend using [np](https://github.com/sindresorhus/np) to publish the package. Running it will involve running `test --prerelease`, and the
147159
versioning step will update the api report + update the version in `src/version.ts`.

etc/astra-db-ts.api.md

Lines changed: 69 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -120,6 +120,7 @@ export class AstraDbAdmin extends DbAdmin {
120120
db(): Db;
121121
drop(options?: AdminBlockingOptions): Promise<void>;
122122
dropNamespace(namespace: string, options?: AdminBlockingOptions): Promise<void>;
123+
findEmbeddingProviders(options?: WithTimeout): Promise<FindEmbeddingProvidersResult>;
123124
get id(): string;
124125
info(options?: WithTimeout): Promise<FullDatabaseInfo>;
125126
listNamespaces(options?: WithTimeout): Promise<string[]>;
@@ -227,6 +228,14 @@ export class CollectionAlreadyExistsError extends DataAPIError {
227228
readonly namespace: string;
228229
}
229230

231+
// @public
232+
export class CollectionNotFoundError extends DataAPIError {
233+
// @internal
234+
constructor(namespace: string, collectionName: string);
235+
readonly collectionName: string;
236+
readonly namespace: string;
237+
}
238+
230239
// @public
231240
export interface CollectionOptions<Schema extends SomeDoc> {
232241
defaultId?: DefaultIdOptions;
@@ -380,6 +389,7 @@ export class DataAPIDbAdmin extends DbAdmin {
380389
createNamespace(namespace: string, options?: LocalCreateNamespaceOptions): Promise<void>;
381390
db(): Db;
382391
dropNamespace(namespace: string, options?: AdminBlockingOptions): Promise<void>;
392+
findEmbeddingProviders(options?: WithTimeout): Promise<FindEmbeddingProvidersResult>;
383393
listNamespaces(options?: WithTimeout): Promise<string[]>;
384394
}
385395

@@ -407,6 +417,15 @@ export interface DataAPIErrorDescriptor {
407417
readonly message?: string;
408418
}
409419

420+
// @public
421+
export class DataAPIHttpError extends DataAPIError {
422+
// @internal
423+
constructor(resp: FetcherResponseInfo);
424+
readonly body?: string;
425+
readonly raw: FetcherResponseInfo;
426+
readonly status: number;
427+
}
428+
410429
// @public
411430
export type DataAPIHttpOptions = DefaultHttpClientOptions | FetchHttpClientOptions | CustomHttpClientOptions;
412431

@@ -541,6 +560,7 @@ export abstract class DbAdmin {
541560
abstract createNamespace(namespace: string, options?: CreateNamespaceOptions): Promise<void>;
542561
abstract db(): Db;
543562
abstract dropNamespace(namespace: string, options?: AdminBlockingOptions): Promise<void>;
563+
abstract findEmbeddingProviders(options?: WithTimeout): Promise<FindEmbeddingProvidersResult>;
544564
abstract listNamespaces(): Promise<string[]>;
545565
}
546566

@@ -560,12 +580,6 @@ export interface DbSpawnOptions {
560580
token?: string | TokenProvider | null;
561581
}
562582

563-
// @public
564-
export class EmbeddingAPIKeyHeaderProvider extends EmbeddingHeadersProvider {
565-
constructor(apiKey: string | nullish);
566-
getHeaders(): Record<string, string>;
567-
}
568-
569583
// @public
570584
export interface DefaultHttpClientOptions {
571585
client?: 'default';
@@ -654,13 +668,57 @@ export class DevOpsUnexpectedStateError extends DevOpsAPIError {
654668
export interface DropCollectionOptions extends WithTimeout, WithNamespace {
655669
}
656670

671+
// @public
672+
export class EmbeddingAPIKeyHeaderProvider extends EmbeddingHeadersProvider {
673+
constructor(apiKey: string | nullish);
674+
getHeaders(): Record<string, string>;
675+
}
676+
657677
// @public
658678
export abstract class EmbeddingHeadersProvider {
659679
abstract getHeaders(): Promise<Record<string, string>> | Record<string, string>;
660680
// @internal
661681
static parseHeaders(token: unknown): EmbeddingHeadersProvider;
662682
}
663683

684+
// @public
685+
export interface EmbeddingProviderAuthInfo {
686+
enabled: boolean;
687+
tokens: EmbeddingProviderTokenInfo[];
688+
}
689+
690+
// @public
691+
export interface EmbeddingProviderInfo {
692+
displayName: string;
693+
models: EmbeddingProviderModelInfo[];
694+
parameters: EmbeddingProviderParameterInfo[];
695+
supportedAuthentication: Record<string, EmbeddingProviderAuthInfo>;
696+
url: string;
697+
}
698+
699+
// @public
700+
export interface EmbeddingProviderModelInfo {
701+
name: string;
702+
parameters: EmbeddingProviderParameterInfo[];
703+
vectorDimension: number | null;
704+
}
705+
706+
// @public
707+
export interface EmbeddingProviderParameterInfo {
708+
defaultValue: string;
709+
help: string;
710+
name: string;
711+
required: boolean;
712+
type: string;
713+
validation: Record<string, unknown>[];
714+
}
715+
716+
// @public
717+
export interface EmbeddingProviderTokenInfo {
718+
accepted: string;
719+
forwarded: string;
720+
}
721+
664722
// @public
665723
export class FailedToLoadDefaultClientError extends Error {
666724
// @internal
@@ -757,6 +815,11 @@ export class FindCursor<T, TRaw extends SomeDoc = SomeDoc> {
757815
toArray(): Promise<T[]>;
758816
}
759817

818+
// @public
819+
export interface FindEmbeddingProvidersResult {
820+
embeddingProviders: Record<string, EmbeddingProviderInfo>;
821+
}
822+
760823
// @public
761824
export interface FindOneAndDeleteOptions extends WithTimeout {
762825
includeResultMetadata?: boolean;
@@ -1321,7 +1384,6 @@ export interface VectorizeServiceOptions {
13211384
export interface VectorOptions {
13221385
dimension?: number;
13231386
metric?: 'cosine' | 'euclidean' | 'dot_product';
1324-
// @alpha
13251387
service?: VectorizeServiceOptions;
13261388
}
13271389

package.json

Lines changed: 3 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -49,14 +49,10 @@
4949
},
5050
"scripts": {
5151
"lint": "eslint src/* tests/*",
52-
"test": "ts-mocha --paths -p tsconfig.json tests/unit/**/*.test.ts tests/integration/**/*.test.ts tests/integration/**/**/*.test.ts",
53-
"test:all": "env ASTRA_RUN_LONG_TESTS=1 ASTRA_RUN_ADMIN_TESTS=1 ts-mocha --paths -p tsconfig.json tests/unit/**/*.test.ts tests/integration/**/*.test.ts tests/integration/**/**/*.test.ts",
54-
"test:coverage": "nyc npm run test:all -- -b",
55-
"test:types": "tsc --noEmit --skipLibCheck",
56-
"test:prerelease": "npm run lint && npm run test:types && npm run test:all -- -b --exit",
57-
"build": "sh ./scripts/build.sh",
52+
"test": "sh scripts/test.sh",
53+
"build": "sh scripts/build.sh",
5854
"list-embedding-providers": "chmod +x scripts/list-embedding-providers.sh && scripts/list-embedding-providers.sh",
59-
"version": "npm run build && git add src/version.ts etc/astra-db-ts.api.md"
55+
"version": "sh scripts/version.sh"
6056
},
6157
"bugs": {
6258
"url": "https://github.com/datastax/astra-ts-client/issues"

scripts/build.sh

Lines changed: 9 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -7,21 +7,27 @@ rm -rf ./dist
77
node scripts/build-version-file.js > src/version.ts
88

99
# Transpiles the project
10-
yes | npx tsc --project tsconfig.build.json
10+
npx tsc --project tsconfig.build.json
1111

1212
# Replaces alias paths with relative paths (e.g. `@/src/version` -> `../../src/version`)
13-
yes | npx tsc-alias -p tsconfig.build.json
13+
npx tsc-alias -p tsconfig.build.json
1414

1515
# Creates the rollup .d.ts, generates an API report in etc/, and cleans up any temp files
16-
yes | npx api-extractor run -c ./api-extractor.jsonc --local && rm -r ./temp
16+
npx api-extractor run -c ./api-extractor.jsonc --local && rm -r ./temp
1717

1818
# Uses a more succinct licence notice + removes block comments (the rollup .d.ts file already contains the ts-doc)
1919
find ./dist -type f -name '*.js' -exec node scripts/reduce-comments.js {} \;
2020

2121
# Adds the missing license notice to the rollup .d.ts
2222
node scripts/add-license-bumf.js dist/astra-db-ts.d.ts
2323

24+
# Delete the "empty" files where only types were declared
25+
node scripts/del-empty-dist-files.js
26+
2427
# Removes all .d.ts files except the main rollup .d.ts
2528
cd dist || return 1
2629
find . -type f -name '*.d.ts' ! -name 'astra-db-ts.d.ts' -exec rm {} +
2730
cd ..
31+
32+
# Delete any empty leftover directories
33+
find ./dist -type d -empty -delete

scripts/del-empty-dist-files.js

Lines changed: 43 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,43 @@
1+
const fs = require('fs');
2+
const path = require('path');
3+
4+
const targetContent = `"use strict";
5+
// Copyright Datastax, Inc
6+
// SPDX-License-Identifier: Apache-2.0
7+
Object.defineProperty(exports, "__esModule", { value: true });
8+
`;
9+
10+
function deleteEmptyFiles(dirPath) {
11+
fs.readdir(dirPath, (err, files) => {
12+
if (err) {
13+
return console.error('Unable to scan directory: ' + err);
14+
}
15+
16+
files.forEach(file => {
17+
const filePath = path.join(dirPath, file);
18+
19+
fs.stat(filePath, (err, stat) => {
20+
if (err) {
21+
return console.error('Error stating file: ' + err);
22+
}
23+
24+
if (stat.isDirectory()) {
25+
deleteEmptyFiles(filePath);
26+
return;
27+
}
28+
29+
fs.readFile(filePath, 'utf8', (err, data) => {
30+
if (err) {
31+
return console.error('Error reading file: ' + err);
32+
}
33+
34+
if (data === targetContent) {
35+
fs.unlink(filePath, (err) => err && console.error('Error deleting file: ' + err));
36+
}
37+
});
38+
});
39+
});
40+
});
41+
}
42+
43+
deleteEmptyFiles('./dist');

0 commit comments

Comments
 (0)