Recently, the database needs to be used in business needs. In the process of looking at the source code of vscode, it is found that vscode uses two databases to store data, SQLite and IndexedDB. This article mainly explains IndexedDB.

Introduction

Around 2008, websites, forums, and social networks began to develop rapidly. Traditional relational databases encountered great challenges in storing and processing data, which are mainly reflected in the following points:

• It is difficult to cope with tens of thousands of high concurrent data writes per second.
• The speed of querying hundreds of millions of data is extremely slow.
• After the sub-library formed by sub-database and sub-table reaches a certain scale, it is difficult to expand further.
• The rules for sub-database and sub-table may change due to changes in requirements.
• Modifying the table structure is difficult.

In many Internet application scenarios, the query requirements for data joint tables are not so strong, and it is not necessary to read the data immediately after writing, but there are very high requirements for the speed of data reading and concurrent writing. Under such circumstances, non-relational databases have developed rapidly.

Relational Database:

• Oracle
• Mysql
• ...

Non-relational database:

• MongoDB
• Redis
• indexedDB
• ...

background

As the functionality of browsers continues to increase, more and more websites are beginning to consider storing a large amount of data on the client, which can reduce the need to obtain data from the server and directly obtain data from the local.

• The storage capacity of Cookies is too small, only 4kb of content can be stored, and every time you interact with the server, the Cookies under the same domain will be carried to the server, and there is no mechanism for related query and conditional query.
• The storage capacity of LocalStorage is also very small, probably no more than 10M. It saves data in the form of key-value pairs, and there is no mechanism for associated query or conditional query.
• The biggest problem with SessionStorage is that every time the application is closed, its contents will be cleared. If you want to store data persistently, you don't need to consider it.
• All the features of WebSql are very good, but this technology has been rejected by the W3C committee. I don’t know when Electron will not support it, and I will be dumbfounded by then.

Therefore, a new solution is needed, which is the background of the birth of IndexedDB.

In layman's terms, IndexedDB is a local database provided by the browser, a JavaScript-based object-oriented database that can be created and manipulated by web scripts. IndexedDB allows storing large amounts of data, providing a lookup interface, and building indexes. These are not available in LocalStorage. As far as the database type is concerned, IndexedDB is not a relational database (does not support SQL query statements), but is closer to a NoSQL database.

features

IndexedDB has the following characteristics.

(1) Key-value pair storage. IndexedDB internally uses an object store (object store) to store data. All types of data can be stored directly, including JavaScript objects. In the object warehouse, data is stored in the form of "key-value pairs". Each data record has a corresponding primary key. The primary key is unique and cannot be duplicated, otherwise an error will be thrown.

(2) Asynchronous. IndexedDB will not lock the browser during operation, and users can still perform other operations, which is in contrast to LocalStorage, whose operations are synchronous. The asynchronous design is to prevent the reading and writing of large amounts of data and slow down the performance of web pages.

(3) Support affairs. IndexedDB supports transactions, which means that as long as one step fails in a series of operation steps, the entire transaction will be cancelled, and the database will be rolled back to the state before the transaction occurred, and there will be no situation where only part of the data is rewritten.

(4) Same-origin restrictions IndexedDB is subject to same-origin restrictions, and each database corresponds to the domain name where it was created. A webpage can only access databases under its own domain name, but not cross-domain databases.

(5) Large storage space The storage space of IndexedDB is much larger than that of LocalStorage. Generally speaking, it is not less than 250MB, and there is even no upper limit.

(6) Support binary storage. IndexedDB can store not only strings, but also binary data (ArrayBuffer objects and Blob objects).

use

• Database: IDBDatabase object
• Object store: IDBObjectStore object
• index: IDBIndex object
• Transaction: IDBTransaction object
• Operation request: IDBRequest object
• Pointer: IDBCursor object
• Primary key collection: IDBKeyRange object

We encapsulate it in the form of a class:

export class IndexedDB {

	static async create(name: string, version: number | undefined, stores: string[]): Promise<IndexedDB> {
		const database = await IndexedDB.openDatabase(name, version, stores);
		return new IndexedDB(database, name);
	}

	static async openDatabase(name: string, version: number | undefined, stores: string[]): Promise<IDBDatabase> {
		mark(`code/willOpenDatabase/${name}`);
		try {
			return await IndexedDB.doOpenDatabase(name, version, stores);
		} catch (err) {
			if (err instanceof MissingStoresError) {
				console.info(`Attempting to recreate the IndexedDB once.`, name);

				try {
					// Try to delete the db
					await IndexedDB.deleteDatabase(err.db);
				} catch (error) {
					console.error(`Error while deleting the IndexedDB`, getErrorMessage(error));
					throw error;
				}

				return await IndexedDB.doOpenDatabase(name, version, stores);
			}

			throw err;
		} finally {
			mark(`code/didOpenDatabase/${name}`);
		}
	}

	private static doOpenDatabase(name: string, version: number | undefined, stores: string[]): Promise<IDBDatabase> {
		return new Promise((c, e) => {
			const request = window.indexedDB.open(name, version);
			request.onerror = () => e(request.error);
			request.onsuccess = () => {
				const db = request.result;
				for (const store of stores) {
					if (!db.objectStoreNames.contains(store)) {
						console.error(`Error while opening IndexedDB. Could not find '${store}'' object store`);
						e(new MissingStoresError(db));
						return;
					}
				}
				c(db);
			};
			request.onupgradeneeded = () => {
				const db = request.result;
				for (const store of stores) {
					if (!db.objectStoreNames.contains(store)) {
						db.createObjectStore(store);
					}
				}
			};
		});
	}

	private static deleteDatabase(indexedDB: IDBDatabase): Promise<void> {
		return new Promise((c, e) => {
			// Close any opened connections
			indexedDB.close();

			// Delete the db
			const deleteRequest = window.indexedDB.deleteDatabase(indexedDB.name);
			deleteRequest.onerror = (err) => e(deleteRequest.error);
			deleteRequest.onsuccess = () => c();
		});
	}

	private database: IDBDatabase | null = null;
	private readonly pendingTransactions: IDBTransaction[] = [];

	constructor(database: IDBDatabase, private readonly name: string) {
		this.database = database;
	}

	hasPendingTransactions(): boolean {
		return this.pendingTransactions.length > 0;
	}

	close(): void {
		if (this.pendingTransactions.length) {
			this.pendingTransactions.splice(0, this.pendingTransactions.length).forEach(transaction => transaction.abort());
		}
		if (this.database) {
			this.database.close();
		}
		this.database = null;
	}

	runInTransaction<T>(store: string, transactionMode: IDBTransactionMode, dbRequestFn: (store: IDBObjectStore) => IDBRequest<T>[]): Promise<T[]>;
	runInTransaction<T>(store: string, transactionMode: IDBTransactionMode, dbRequestFn: (store: IDBObjectStore) => IDBRequest<T>): Promise<T>;
	async runInTransaction<T>(store: string, transactionMode: IDBTransactionMode, dbRequestFn: (store: IDBObjectStore) => IDBRequest<T> | IDBRequest<T>[]): Promise<T | T[]> {
		if (!this.database) {
			throw new Error(`IndexedDB database '${this.name}' is not opened.`);
		}
		const transaction = this.database.transaction(store, transactionMode);
		this.pendingTransactions.push(transaction);
		return new Promise<T | T[]>((c, e) => {
			transaction.oncomplete = () => {
				if (isArray(request)) {
					c(request.map(r => r.result));
				} else {
					c(request.result);
				}
			};
			transaction.onerror = () => e(transaction.error);
			const request = dbRequestFn(transaction.objectStore(store));
		}).finally(() => this.pendingTransactions.splice(this.pendingTransactions.indexOf(transaction), 1));
	}

	async getKeyValues<V>(store: string, isValid: (value: unknown) => value is V): Promise<Map<string, V>> {
		if (!this.database) {
			throw new Error(`IndexedDB database '${this.name}' is not opened.`);
		}
		const transaction = this.database.transaction(store, 'readonly');
		this.pendingTransactions.push(transaction);
		return new Promise<Map<string, V>>(resolve => {
			const items = new Map<string, V>();

			const objectStore = transaction.objectStore(store);

			// Open a IndexedDB Cursor to iterate over key/values
			const cursor = objectStore.openCursor();
			if (!cursor) {
				return resolve(items); // this means the `ItemTable` was empty
			}

			// Iterate over rows of `ItemTable` until the end
			cursor.onsuccess = () => {
				if (cursor.result) {

					// Keep cursor key/value in our map
					if (isValid(cursor.result.value)) {
						items.set(cursor.result.key.toString(), cursor.result.value);
					}

					// Advance cursor to next row
					cursor.result.continue();
				} else {
					resolve(items); // reached end of table
				}
			};

			// Error handlers
			const onError = (error: Error | null) => {
				console.error(`IndexedDB getKeyValues(): ${toErrorMessage(error, true)}`);

				resolve(items);
			};
			cursor.onerror = () => onError(cursor.error);
			transaction.onerror = () => onError(transaction.error);
		}).finally(() => this.pendingTransactions.splice(this.pendingTransactions.indexOf(transaction), 1));
	}
}

use in service

import { IndexedDB } from 'vs/base/browser/indexedDB'; //introduce

private readonly whenConnected!: Promise<IndexedDB>; //Define a private variable to store the database

Initialize the constructor

this.whenConnected = this.connect(); //link database

private async connect(): Promise<IndexedDB> {
	try {
		return await IndexedDB.create('indexedDB-test', undefined, ['test-store1']);
	} catch (error) {
		throw error;
	}
}

used in the method

let indexedDB = await this.whenConnected;

// increase
try {
	await indexedDB.runInTransaction('test-store1', 'readwrite', store => store.add({ 'test': '222' }, 'key3'));
} catch (e) {
	console.log('Error storing data')
}

// delete
try {
	await indexedDB.runInTransaction('test-store1', 'readwrite', store => store.delete('key3'));
} catch (e) {
	console.log('Error deleting data');
}

// change
try {
	await indexedDB.runInTransaction('test-store1', 'readwrite', store => store.put({ 'lichangwei': '123' }, 'key3'));
} catch (e) {
	console.log('Error deleting data');
}

// check
const value = await indexedDB.runInTransaction('test-store1', 'readwrite', store => store.getAll());

If you have any questions, please leave a message

Refer to Ruan Yifeng's log