A powerful PHP NoSQL document store built on top of SQLite.
- About DocLite
- Getting Started
- The Database
- Collections
- Documents
- Other info
DocLite is a powerful NoSQL document store for PHP built on top of SQLite. It uses the PHP PDO SQLite library to access a SQLite database and automatically manage documents organized in to named collections, which are stored as JSON.
DocLite takes advantage of the SQLite JSON1 extension (this is usually bundled in to the libsqlite included with your PHP distribution, so you probably already have it) to store, parse, index and query JSON documents - giving you the power and flexibility of a fully transactional and ACID compliant NoSQL solution, yet contained within the local file system. No need for more complex systems like Mongo, CouchDB or Elasticsearch when your requirements are slim. No need for any external dependencies, just PHP with PDO SQLite enabled.
DocLite provides a simple, intuitive, flexible and powerful PHP library that you can learn, install and start using in minutes.
DocLite lends itself well to a variety of use cases, including but not limited to:
-
Agile development and rapid prototyping while your requirements are evolving.
-
Powerful, self-contained NoSQL database for small to medium websites or applications, such as blogs, business website, CMS, CRM or forums.
-
A fast and reliable cache for data retrieved from remote databases, APIs or servers. Process your data in to documents, save in DocLite and easily query and filter your data as needed.
-
Robust, performant, ACID compliant replacement for weaker, slower, flat-file data stores utilizing JSON, XML or YAML.
-
Application database for web apps installed and run on a local environment.
-
Database for microservices and middleware.
-
Fast in-memory database for data processing or machine learning algorithms.
Broadly speaking, DocLite is suitable for the same uses cases as the underlying SQLite engine it is built on, but where you desire a NoSQL solution.
System requirements
-
PHP 7.4 or above
-
With PDO SQLite enabled, built against libsqlite ≥ 3.18.0 with JSON1 extension.
(on most systems, if you're running PHP 7.4 you probably already meet the second requirement)
Usage Overview
DocLite provides both a FileDatabase and MemoryDatabase implementation.
To create or open an existing database, simply create a Database object, specifying the file path if using a FileDatabase.
If your FileDatabase does not exist, it will be created (ensure your script has the appropriate write permissions).
This will include creating any parent directories as required.
If you specify an existing directory without a filename, a default filename data.db will be used.
use Gebler\Doclite\{FileDatabase, MemoryDatabase};
// To create or open an existing file database.
$db = new FileDatabase('/path/to/db');
// To open an existing file database in read-only mode.
$db = new FileDatabase('/path/to/existing/db', true);
// To create a new in-memory database.
$db = new MemoryDatabase();Once you have opened a database, you can obtain a document Collection which will be automatically created
if it does not exist.
$users = $db->collection("user"); The Collection object can then be used to retrieve, create and manipulate documents.
// Create a new User in the collection
$user = $users->get();
// Get the automatically generated document ID
$id = $user->getId();
// Set properties by magic set* methods
$user->setUsername("dwgebler");
$user->setRole("admin");
$user->setPassword(password_hash("admin", \PASSWORD_DEFAULT));
$user->setCreated(new \DateTimeImmutable);
// Update the user in the collection
$user->save();
// Retrieve this user later on by ID
$user = $users->get($id);
// Or search for a user by any field
$user = $users->findOneBy(["username" => "dwgebler"]);In the example above, $user is an instance of a DocLite Document, but you can also
hydrate objects of your own custom classes from a collection.
class CustomUser
{
private $id;
private $username;
private $password;
public function getId() {...}
public function setId($id) {...}
public function getUsername() {...}
public function setUsername($username) {...}
}
// Retrieve a previously created user and map the result on to a CustomUser object.
// You can also pass a null ID as the first parameter to create a new CustomUser.
$user = $users->get($id, CustomUser::class);
// $user is now an instance of CustomUser and can be saved through the Collection.
$users->save($user);To learn more about the Collection object including how to query a document store, please read the full
documentation below.
DocLite is built on top of SQLite 3 and supports two types of database; file and memory.
The corresponding classes are FileDatabase and MemoryDatabase.
MemoryDatabase is stored in volatile memory and is therefore ephemeral for the lifetime
of your application scripts. Its constructor takes optional parameters:
- a boolean flag indicating whether to enable full text search features (defaults to
false) - this feature requires SQLite to have been compiled with the FTS5 extension. - an integer representing the maximum connection timeout in seconds (defaults to
1) which is how long the connection should wait if the underlying SQLite database is locked. - A PSR-3 compatible logger instance (defaults to
null).
```php
use Gebler\Doclite\MemoryDatabase;
$db = new MemoryDatabase();
// With full text search enabled and a 2-second connection timeout
$logger = new \Monolog\Logger('my-logger');
$db = new MemoryDatabase(true, 2, $logger); FileDatabase constructor takes one mandatory and then some optional parameters;
only the file or directory path to a new or existing database is required.
Optional parameters are:
- a boolean flag indicating whether the database should be opened in read-only mode, which defaults to
false. - a boolean flag indicating whether to enable full text search features (defaults to
false) - this feature requires SQLite to have been compiled with the FTS5 extension. - an integer representing the maximum connection timeout in seconds (defaults to
1) which is how long the connection should wait if the underlying SQLite database is locked. - A PSR-3 Logger instance to use for logging database events.
The path supplied to FileDatabase can be a relative or absolute path which is any of:
- An existing directory with read and write access.
- A non-existent file in a directory with read-write access.
- An existing database in a directory with read-write or read-only access (read-only mode).
- A non-existing directory path which your script has permission to create.
If no file name is specified, a default file name data.db will be used for the underlying database.
use Gebler\Doclite\FileDatabase;
// Open a new database
$db = new FileDatabase('./data/mydb.db');
// Open an existing database in read-only mode
$db = new FileDatabase('./data/mydb.db', true);
// Open a new database called data.db in existing directory /home/data
$db = new FileDatabase('/home/data');
// All options - path, read-only mode, full text search, connection timeout and logger
$logger = new \Monolog\Logger('mylogger');
$db = new FileDatabase('./data/mydb.db', false, true, 1, $logger);
// Or, in PHP 8, named parameters:
$db = new FileDatabase(path: './data/mydb.db', readOnly: true, ftsEnabled: true);If you open a database in read-only mode, you will be able to retrieve documents from a collection, but you will not be able to save them or create new documents or collections. Attempting to do so will trigger an error.
It is good practice wrapping FileDatabase creation in a try-catch block.
Initializing a FileDatabase may throw either an IOException (for errors relating to the file system)
or a DatabaseException (for errors establishing the DB connection).
use Gebler\Doclite\Exception\IOException;
use Gebler\Doclite\Exception\DatabaseException;
use Gebler\Doclite\FileDatabase;
try {
$db = new FileDatabase('/path/to/db');
} catch (IOException $e) {
var_dump($e->getMessage());
} catch (DatabaseException $e) {
var_dump($e->getMessage());
}To enable logging queries and parameters in the full and final SQL sent to the database, pass a PSR-3 logger instance in to you FileDatabase or MemoryDatabase
either via the constructor or the $database->setLogger(LoggerInterface $logger) method at any time.
Then call $database->enableQueryLogging() to enable logging of all queries. These will be logged at the debug level.
Or call $database->enableSlowQueryLogging() to enable logging of queries that take longer than 500ms.
These will be logged with a warning level.
As long as a LoggerInterface instance is set on a database, any exceptions will also be logged at the error level.
You can disable logging by calling $database->disableQueryLogging() or $database->disableSlowQueryLogging().
DocLite primarily throws a DatabaseException when any error occurs. This is true across
the Database, Collection and Document types. A Database exception will
include a message, an error code (see below), any underlying system exception if there was one (so normal
Exception behaviour up to this point), plus any SQL query which was being executed
(DocLite hides these from you during normal operation, of course, as it is a NoSQL solution,
but they are useful for filing bug reports!), and an array of any relevant parameters - these
may be things like a document ID, the document data, etc.
use Gebler\Doclite\Exception\DatabaseException;
...
try {
$user->setUsername("dwgebler");
$user->save();
} catch (DatabaseException $e) {
var_dump($e->getMessage(), $e->getCode(), $e->getQuery(), $e->getParams());
}A DatabaseException can occur on any Database, Collection or Document
method which interacts with the underlying database.
Error codes are represented by public constants in the DatabaseException class.
The full list of error codes are as follows:
| Constant | Meaning |
|---|---|
| ERR_COLLECTION_IN_TRANSACTION | Attempted to begin, rollback or commit on a collection while a transaction on a different collection was already in progress. |
| ERR_CONNECTION | Unable to connect to database |
| ERR_NO_SQLITE | PDO SQLite extension is not installed |
| ERR_NO_JSON1 | SQLite does not have the JSON1 extension installed |
| ERR_NO_FTS5 | FTS5 extension not installed |
| ERR_INVALID_COLLECTION | Invalid collection name |
| ERR_MISSING_ID_FIELD | Custom class for mapping a document does not have an ID field |
| ERR_INVALID_FIND_CRITERIA | Attempted to find a document by non-scalar value |
| ERR_INVALID_ID_FIELD | Specified unique ID field for custom class does not exist, nor does default |
| ERR_ID_CONFLICT | Multiple documents in the same collection have the same ID |
| ERR_CLASS_NOT_FOUND | Custom class name being used for a document does not exist |
| ERR_INVALID_UUID | Attempted to get the timestamp from an invalid UUID |
| ERR_QUERY | Error executing SQL query |
| ERR_READ_ONLY_MODE | Attempted a write operation on a read only database |
| ERR_INVALID_JSON_SCHEMA | Attempted to import an invalid JSON schema |
| ERR_INVALID_DATA | Data does not match loaded JSON schema |
| ERR_MAPPING_DATA | Unable to map document to class |
| ERR_IMPORT_DATA | Error importing data |
| ERR_IN_TRANSACTION | Attempting locking operation while in a transaction |
| ERR_INVALID_TABLE | Attempting to access invalid table |
| ERR_UNIQUE_CONSTRAINT | Attempting to insert a document with a unique field that already exists |
DocLite can import data from and export data to JSON, YAML, XML and CSV files.
For this purpose, the Database object provides two methods, import() and
export().
⚠️ Import or export operations on very large collections may exhaust memory. This feature will (probably) be improved and made more efficient for working with large data sets in future.It is recommended to use JSON for exports you intend to reload in to a DocLite database. Support for other formats is experimental.
The data you want to import can be organized either in to files, where each file represents a collection of multiple documents, or a directory where each sub-directory represents a collection and contains a number of files each representing a single document.
import(string $path, string $format, int $mode)
format can be any of json, yaml, xml, or csv. This should also
match the extension of the filename(s) containing your data.
When using the csv format, the first line of a CSV file is assumed to be a
header line containing field names.
mode can be either of the constants Database::MODE_IMPORT_COLLECTIONS or
Database::MODE_IMPORT_DOCUMENTS.
Collection names are inferred from the subdirectory or file names. For example,
/path/to/collections/users.json will import to the users collection, as will
a sub-directory /path/to/collections/users/ when importing a collection from
multiple files.
// Create a new, empty database
$db = new FileDatabase('/path/to/data.db');
// Import the contents of a directory where each file is a collection
$db->import('/path/to/collections', 'json', Database::IMPORT_COLLECTIONS);
// Import the contents of a directory where each sub directory is a collection
// of files representing single documents.
$db->import('/path/to/collections', 'json', Database::IMPORT_DOCUMENTS);When you import documents in to a collection, any documents which have a unique ID matching an existing document in the database will overwrite that document. Otherwise, new documents will be created for any documents with an unmatched or missing ID.
💡 Each Collection import will be wrapped in a single transaction, so imports are atomic per collection. You can also speed up bulk imports by setting the advanced options (see below) to alter the database's synchronization and rollback journal modes to something a little more permissive, if you understand the implications of doing so.
You can export the entire contents of one or more collections. Much like importing data, you can choose whether DocLite should export this as one file per collection containing multiple documents, or one directory per collection with one file per document.
export(string $path, string $format, int $mode, array $collections = [])
format can be any of json, yaml, xml, or csv.
mode can be either of the constants Database::MODE_EXPORT_COLLECTIONS or
Database::MODE_EXPORT_DOCUMENTS.
collections can be a mix of strings of collection names and/or Collection
objects. If this is empty, all collections in the database will be exported.
// Export the entire database to one file per collection in the specified
// output directory.
$db->export('/path/to/export', 'json', Database::EXPORT_COLLECTIONS);
// Export the entire database to a directory structure with one file per document.
$db->export('/path/to/export', 'json', Database::EXPORT_DOCUMENTS);
// Export only the "User" and "Person" collections.
// Assume Collection $persons = $db->get("Person");
$db->export(
'/path/to/export',
'json',
Database::EXPORT_COLLECTIONS,
['User', $persons]
);
⚠️ The XML standard imposes some restrictions on entity names. When exporting to this format, DocLite will replace any invalid characters in document fields with underscores. This means you may not be able to recreate your document store exactly as it was should you subsequently import these files in to a DocLite database.
DocLite Database objects have a few methods for more advanced options.
// Return the version of DocLite as a SemVer string, e.g. 1.0.0
$db->getVersion();Call $db->optimize() to attempt database optimization. This function does
not return anything, though can throw a DatabaseException if something goes
wrong. Periodic optimization can reduce database file size and improve performance.
The underlying SQLite sync mode can be set to one of the following constants in
the Database class.
See SQLite documentation
for details of the implications of changing this value; disabling sync can lead
to data loss in the event of a crash or power loss.
| Constant | Meaning |
|---|---|
| MODE_SYNC_OFF | Disable sync |
| MODE_SYNC_NORMAL | Normal sync Default setting |
| MODE_SYNC_FULL | Full sync |
| MODE_SYNC_EXTRA | Extra sync |
Call $db->setSyncMode(Database::MODE_CONSTANT) to set the mode.
For example to set Full Sync mode, call $db->setSyncMode(Database::MODE_SYNC_FULL).
This function returns true on success or false on failure.
Call $db->getSyncMode() to get the current mode which can be compared to one
of the constants. The return type is int.
The underlying SQLite management of the rollback journal can be set to one of the following constants. See SQLite documentation for the implications of changing this value; disabling the rollback journal can lead to unintended data state.
⚠️ Warning: If you disable the rollback journal, transactions, atomic commits and rollbacks will no longer work. The behaviour of transaction methods on a collection in this mode is undefined and may lead to unpredictable results or data corruption. You should therefore not use the transaction methods in MODE_JOURNAL_NONE.
| Constant | Meaning |
|---|---|
| MODE_JOURNAL_NONE | Disable the rollback journal |
| MODE_JOURNAL_MEMORY | In-memory rollback journal only |
| MODE_JOURNAL_WAL | Use the write ahead log Default setting |
| MODE_JOURNAL_DELETE | Delete rollback journal at end of each transaction |
| MODE_JOURNAL_TRUNCATE | Truncate rollback journal at end of each transaction |
| MODE_JOURNAL_PERSIST | Prevent the rollback journal being deleted |
Call $db->setJournalMode(Database::MODE_CONSTANT) to set the mode.
For example to set WAL mode, call $db->setJournalMode(Database::MODE_JOURNAL_WAL).
This function returns true on success or false on failure.
Call $db->getJournalMode() to get the current mode which can be compared to one
of the constants. The return type is string.
Collections are at the heart of DocLite. A Collection represents a named group of documents
(for example, "Users") and is analogous to a table in a structured database.
💡 Note: Collections are represented in the underlying SQLite database as tables. They must therefore obey a few rules:
- Collection names cannot start with
sqlite_- Collection names cannot start with a number.
- Collection names may contain only alphanumeric characters and underscores.
- Collection names cannot be longer than 64 characters.
A Collection object is the means by which you create, find, update and delete documents.
Every document in a collection must have a unique ID. You can either supply this yourself, or one will be created for you when you first instantiate a document. Auto generated IDs take the form of a v1 UUID which includes a timestamp of when the document was first created.
Collections are obtained from a FileDatabase or MemoryDatabase by calling the collection
method. If the collection does not exist, it will be automatically created.
$userCollection = $db->collection("Users");Once you have a collection, create a new document by calling the collection's get method.
$newUser = $userCollection->get();By default, documents are returned as a DocLite Document object, which provides
a save() method. You can also save a document of any type by calling save()
on the collection with the document object as a parameter.
// works for DocLite Document objects
$newUser->save();
// works for both DocLite documents and documents mapped to custom types
$userCollection->save($newUser);get can also be used to retrieve a document by its ID.
$existingUser = $userCollection->get($id);By default, retrieving a document will return a DocLite Document object, which provides magic
methods and properties for you to access and manipulate the document data. It is however also
possible to create or retrieve a document as an object of any custom class, provided that class
has either public properties or getter/setter methods for the document fields you wish to hydrate.
// Get a user as an object of type CustomUser.
$user = $userCollection->get($id, CustomUser::class);By default, DocLite will look for a property called id to populate with the document's unique id.
If you want to use a different property on a custom class to store this id, for example because your class
does not have an id property, or you are using it for something else, you can specify a custom ID property
name as a third parameter to get.
$user = $userCollection->get($id, CustomUser::class, 'databaseId');Alternatively, you can add a public property or getter/setter to your class called docliteId
and DocLite will automatically attempt to populate this instead in the absence of an id property.
While the Document class provides a built-in save() method as a convenience to update a Document
in storage, documents represented as your own custom classes must be saved through the collection object.
$userCollection->save($user);If you are using a custom property on your class to hold the document's unique ID, you should supply the ID as an additional parameter.
$userCollection->save($user, $user->getDatabaseId());Finally, when saving a document represented as a custom class, you can specify an optional third parameter to list any properties on the object you do not want to be stored in the document. It is only necessary to do this either for properties you wish to be excluded which are public / have getter/setter methods, or public get methods which do not represent properties.
$userCollection->save($user, $user->getDatabaseId(), ['nonDatabaseField']);Much like save(), there is both a convenience delete() method on DocLite
Document objects and a deleteDocument(object $document) method on the
collection itself.
// Works for DocLite Document objects.
$user->delete();
// works for both DocLite documents and documents mapped to custom types
$userCollection->deleteDocument($user);The Collection object provides a range of methods to find documents by arbitrary criteria.
Find a single document where all keys match the specified values by calling findOneBy.
$user = $userCollection->findOneBy([
'role' => 'admin',
'name' => 'Mr Administrator',
]);findOneBy takes optional custom class name and custom class ID field parameters in the same
manner as get.
$user = $userCollection->findOneBy(['username' => 'admin'], CustomUser::class, 'databaseId');If a document which matches the criteria cannot be found, null is returned.
The function findAllBy works the same way as findOneBy but will return a
generator which you can iterate over, or convert to array via PHP's
iterator_to_array function.
foreach($userCollection->findAllBy(['active' => true]) as $user) {
...
}To retrieve all documents in a collection, use findAll(). Like the previous two functions,
findAll can take an optional custom classname and ID property as parameters.
foreach($userCollection->findAll() as $user) {
...
}DocLite includes a powerful query building mechanism to retrieve or delete all documents in a collection matching arbitrary criteria.
To build a query, use any combination of the where(), and(), or(), limit(),
offset() and orderBy() functions on the collection object, followed by a
call to fetch(), delete() or count().
You can also run nested queries to group clauses together via union()
(for grouping clauses by OR) and intersect() (for grouping clauses by AND).
You can query a document to any depth by separating nested fields with a . dot character,
you can also add square brackets [] to the end of a field which is a list to query
all the values inside that list for any match.
The advanced queries APIs are better understood by example.
For the following code snippets, imagine each document of your user collection looks like the following data example, expressed here as YAML:
Sample user document
username: adamjones
first_name: Adam
last_name: Jones
password: "$2y$10$LRS.0xUCJjWSmQuWMMRsuurZ0OGlU.NH7KYXsipzkfUa0YREEarj2"
address:
street: 123 Fake Street
area: Testville
county: Testshire
postcode: TE1 3ST
roles:
- USER
- EDITOR
telephone: "+441234567890"
registered: true
active: true
lastLogin: "2021-02-13T10:34:40+00:00"
email: [email protected]
api_access:
"/v1/pages/":
- POST
- GET
"/v1/contributors/":
- GETHere are some example queries you could run against a collection of these documents.
Sample queries
$users = $db->collection("Users");
$activeUsers = $users->where('active', '=', true)->fetch();
$gmailUsers = $users->where('email', 'ENDS', '@gmail.com')->fetch();
$registeredAndNotActiveUsers = $users->where('registered', '=', true)
->and('active', '=', false)
->fetch();
$usersInPostalArea = $users->where('address.postcode', 'STARTS', 'TE1')->fetch();
$usersWith123InPhone = $users->where('telephone', 'CONTAINS', '123')->fetch();
$usersWithNoNumbersInUsername = $users->where('username', 'MATCHES', '^[A-Za-z]*$')
->fetch();
$usersWithEditorRole = $users->where('roles[]', '=', 'EDITOR')->fetch();
$usersWithEditorOrAdminRole = $users->where('roles[]', '=', 'ADMIN')
->or('roles[]', '=', 'EDITOR')
->fetch();
$usersWithEditorAndAdminRole = $users->where('roles', '=', ['ADMIN', 'EDITOR']);
$usersWhoHaveAtLeastOneRoleWhichIsNotAdmin = $users->where('roles[]', '!=', 'ADMIN')->fetch();
/*
* This next one is trickier. "roles" is a list of values in our document.
* As we can see above, roles[] != ADMIN would return all users who
* have at least one role in their list which is not ADMIN.
* But this means if a user has roles ["USER","ADMIN"], they would
* be matched.
* So for users who do NOT have the ADMIN role at all, we can
* quote the value "ADMIN" and ask for matches where the entire list of roles
* (so no square brackets) does not contain this value.
*/
$usersDoNotHaveAdminRole = $users->where('roles', 'NOT CONTAINS', '"ADMIN"')->fetch();
$deleteAllUsersWithEditorRole = $users->where('roles[]', '=', 'EDITOR')->delete();
$first10UsersOrderedByFirstName = $users->orderBy('first_name', 'ASC')
->limit(10)
->fetch();
$next10UsersOrderedByFirstName = $users->orderBy('first_name', 'ASC')
->limit(10)
->offset(10)
->fetch();
// Use [] on any field which is a list to search within its sub-items
$usersWithPostAccessToPagesApi = $users->where(
'api_access./v1/pages/[]', '=', 'POST')->fetch();
$allUsersWithPostAccessToAnyApi = $users->where('api_access[]', '=', 'POST')
->fetch();
$start = new DateTimeImmutable('2021-03-01 00:00:00');
$end = new DateTime('2021-06-30');
$usersWhoSignedUpBetweenMarchAndJune = $users->where('date', 'BETWEEN', $start, $end)->fetchArray();
/**
* Nested queries are also possible.
* To get all users where
* (active=true and address.postcode matches '^[A-Za-z0-9 ]*$')
* OR
* (roles[] list contains "EDITOR" and lastLogin > 2021-01-30)
*/
$nestedUsers = $users->where('active', '=', true)
->and('address.postcode', 'MATCHES', '^[A-Za-z0-9 ]*$')
->union()
->where('roles[]', '=', 'EDITOR')
->and('lastLogin', '>', '2021-01-30')
->fetch();💡 Like
findAllBy(), thefetch()method returns a generator, not an array. If you would like all results at once, replacefetch()withfetchArray().
💡 The
fetch()method on advanced queries can take a custom class name and custom ID field as optional parameters, just like thefindOneBy,findAllByandfindAll()methods.
💡 Speed up complex queries by enabling DocLite's caching feature.
Advanced queries support the following operators:
| Operator | Meaning |
|---|---|
| = | Equals, exact match |
| != | Not equals |
| < | Less than |
| > | Greater than |
| <= | Less than or equal |
| >= | Greater than or equal |
| BETWEEN | Between two values, inclusive. Equivalent to >= AND <= |
| NOT BETWEEN | Not between two values, inclusive. Equivalent to < OR > |
| STARTS | Text starts with |
| NOT STARTS | Text does not start with |
| ENDS | Text ends with |
| NOT ENDS | Text does not end with |
| CONTAINS | Text contains |
| NOT CONTAINS | Text does not contain |
| MATCHES | Text regular expression match |
| NOT MATCHES | Text negative regular expression match |
| EMPTY | Has no value, null |
| NOT EMPTY | Has any value, not null |
It is possible to join a collection to one or more other collections when running a query, to include matching results from these collections in the documents returned. This works much like a foreign key in a relational database.
For example, if you have a users collection and a comments collection, where some documents in
comments contain a field user_id. You can query users and join on comments, such that any documents
matching in comments for the same user ID will be included in the users document, under a field called comments.
/**
* Imagine a user document like:
* {"__id":"1", "name":"John Smith"}
*
* and a corresponding comments document like:
* {"__id":"5", "user_id": "1", "comment":"Hello world!"}
*
* You can query the users collection with a join to retrieve an aggregated document like this:
* {"__id":"1","name":"John Smith","comments":[{"__id":"5","comment":"Hello world!"}]}
*/
$users = $db->collection('Users');
$comments = $db->collection("Comments");
$users->where('__id', '=', '1')->join($comments, 'user_id', '__id')->fetchArray();The Collection::join method takes the collection to join as the first parameter, the name of the
document field in that collection to use as a foreign key as the second parameter, and
the corresponding field in documents in the joining collection (e.g. Users) to match against.
The above example therefore is looking for documents in Comments where the field user_id matches the
field __id in Users.
As join is part of the standard query building interface on a Collection, you can combine with other
query operators such as where, and etc. or other joins.
DocLite can cache the results of queries to speed up retrieval of complex result sets. For very simple queries, however, this may provide no benefit or even incur a small performance penalty, so you should only turn it on if you need to.
To turn on caching for a collection, call the collection object's
enableCache() method.
Likewise, you can disable caching by calling disableCache().
Cache results are valid for the cache lifetime, which defaults to 60 seconds. You can
change the cache validity period by calling setCacheLifetime($seconds). A cache
lifetime of zero means cached results will never expire.
You can manually flush the cache by calling clearCache().
$userCollection->enableCache();
// Set the cache validity period to 1 hour.
$userCollection->setCacheLifetime(3600);
$userCollection->disableCache();
$userCollection->clearCache();Finally, the Database object can be set to automatically prune expired
entries whenever the cache is queried. This behaviour is disabled by default;
to enable auto-pruning, call enableCacheAutoPrune() on the database object.
$db->enableCacheAutoPrune();💡 For complex queries, the cache is very fast. If you are running a large number of complex queries on a large data set and these queries are likely to be repeated without the data changing in storage for the lifetime of the cache, it is a good idea to make use of DocLite's caching.
It is possible to build indexes on any document fields inside a collection to speed up queries against that field.
When you create a collection, an index is automatically added for the internal
ID field. To add a custom index, call the addIndex method with the name of a
document field.
$userCollection->addIndex('email');To add a single index on multiple fields (as per a multi-column index),
simply call addIndex with the additional field names as separate
parameters.
$userCollection->addIndex('first_name', 'last_name');You can also add a unique index which acts as a constraint on the field, ensuring that no two documents in the collection can have the same value for that field or field combination.
$userCollection->addUniqueIndex('email');If you attempt to add a document to a collection with a unique index and a value
that already exists in the collection, a DatabaseException will be thrown with the
code DatabaseException::ERR_UNIQUE_CONSTRAINT.
💡 Note: indexes are an advanced feature which work the same way they do in any other SQLite database, the only difference being they are created on document fields rather than a table column. Poorly chosen indexes may provide no benefit or even slow down queries.
To delete all documents in a collection entirely, call the deleteAll() method.
$userCollection->deleteAll();It is possible to wrap a sequence of database operations inside a transaction.
To do this, use the Collection's beginTransaction(), commit() and
rollback() methods.
$collection = $db->collection("Users");
$collection->beginTransaction();
// ...do some stuff, insert a bunch of records or whatever...
// commit the results and end the transaction
$collection->commit();
// or rollback the changes and end the transaction
$collection->rollback();DocLite is able to build powerful full text indexes against collections to allow you to search and produce a list of documents, ordered by relevance, where specified fields match some text or phrase.
Full text search capability requires your PHP's libsqlite to be built with the FTS5 extension.
Just like the JSON1 extension, this is usually bundled in to the standard distribution so you probably already have it.
To search a collection, ensure you have initialized your Database with the full text parameter set to true to enable
this feature, then simply call the search() method on any collection, with the search phrase followed by an array of
the names of any document fields you wish to search against.
$path = '/path/to/db';
$readOnly = false;
$ftsEnabled = true;
$timeout = 1;
$db = new FileDatabase($path, $readOnly, $ftsEnabled, $timeout);
$blogPosts = $db->collection("posts");
$results = $blogPosts->search('apache', ['title', 'summary', 'content']);Results are automatically ordered by relevance.
💡 DocLite will intelligently manage your full text indexes to keep your database optimized. When you call
search(), if there is no index for the set of fields you are searching on, it will be created automatically on the first search. If you later callsearch()on a superset of fields for an existing index, the original index will be destroyed and a new, larger index encompassing all searched fields created. This is so DocLite can use the smallest possible index for all the fields you wish to search against.On small collections, this process is so fast you may not see an impact. If, however, you have a very large collection, the recommendation is to create your full text indexes by calling
search()once from a separate script, so that when your application first runs and callssearch(), the relevant indexes already exist.
Because search() is part of the standard query fetching interface on a collection (same as fetch() and count()), it can be preceded by
normal query filters using where(), and() etc. Similar to fetch(), the search() method returns
a generator. You can convert the results to an array by using PHP's iterator_to_array() function.
Documents are a variadic structured store of data in the form of key-value pairs, stored in the database as JSON; that is, each document inside a collection can have its own freeform structure. It does not matter whether this matches the structure of any other documents in the same collection, that is up to how your application decides to use DocLite.
A document will by default be represented by the DocLite Document class, however it is also
possible to create or retrieve documents from a collection and map them on to your own classes.
See the Collection documentation for more details on this.
The Document class provides magic get and set methods and property accessors for arbitrary document
keys. That is, once you have a Document from a Collection, you can set or read any properties
you like by either method:
$users = $db->collection("Users");
// Create a new Document with an auto generated UUID.
$user = $db->get();
// Create a new property called username via a magic setter.
$user->setUsername('dwgebler');
// Create a new property called password via a magic property.
$user->password = password_hash("admin", \PASSWORD_DEFAULT);
// Read the username property via a magic property.
echo $user->username;
// Read the password property via a magic getter.
echo $user->getPassword();
// Properties can contain scalar values, arrays, or even other Documents and
// custom objects.
$user->setRoles(['user', 'admin']);There is one small semantic difference between the magic method and property access techniques; when using magic methods, the property names are converted from camelCase to snake_case, whereas direct property access is literal e.g.
// setter uses camel case
$user->setFirstName('Dave');
// but the corresponding property created will be lower cased and snake_cased
echo $user->first_name;
// if you want a key in a document to be case sensitive, set it as a property only
$user->FirstName = 'Dave';
// you should now use the property access to retrieve its value later on
echo $user->FirstName;
// This will not work and will raise a ValueError on getFooBar(),
// because the method call will look for a property called foo_bar
$user->FooBar = 'baz';
$user->getFooBar();The Document class also provides two further methods, getValue and setValue,
to query a document by nested keys using a path in dot . notation. These
methods can also be used to get or set fields with names which can't be
expressed through magic set methods or properties.
getValue() raises a ValueError if the specified path cannot be found.
setValue() will automatically create any parent properties on a nested path.
// This is the same as:
// $address = $user->getAddress();
// $postcode = $address['postcode'];
$user->getValue('address.postcode');
// Assume "roles" is a list, this will return an array
$user->getValue('roles');
// Retrieve the first role
$user->getValue('roles.0');
// Assume api_access is a dictionary of keys mapped to lists.
// This will return the list of data under the /v1/users/ key
// as an array.
$access = $user->getValue('api_access./v1/users/');
if (!in_array('POST', $access)) { ... }
// If address does not exist, it will be created with postcode as a key.
$user->setValue('address.postcode', 'TE1 3ST');
// Or set a value with special characters in the name:
$user->setValue('api_access./v1/users/', ['GET', 'POST']);💡 The values of document fields are arbitrary. Scalar values, arrays and even objects of custom classes can all be stored in a document.
If you've retrieved a document as a default Document object, it is still
possible to map document fields which represent custom objects to custom classes.
To do this, use the Document's map() method and pass it a field name (which
can use the nested dot . notation as described above), along with either a
class name or existing object instance if you wish to populate an existing object.
Consider you have the following custom class in your application:
Sample class
class Person
{
private $id;
private $firstName;
private $lastName;
private $address = [];
private $postcode;
private $dateOfBirth;
private $identityVerified;
public function getId(): ?int
{
return $this->id;
}
public function setId(string $id): self
{
$this->id = $id;
return $this;
}
public function getFirstName(): ?string
{
return $this->firstName;
}
public function setFirstName(string $firstName): self
{
$this->firstName = $firstName;
return $this;
}
public function getLastName(): ?string
{
return $this->lastName;
}
public function setLastName(string $lastName): self
{
$this->lastName = $lastName;
return $this;
}
public function getAddress(): ?array
{
return $this->address;
}
public function setAddress(array $address): self
{
$this->address = $address;
return $this;
}
public function getPostcode(): ?string
{
return $this->postcode;
}
public function setPostcode(string $postcode): self
{
$this->postcode = $postcode;
return $this;
}
public function getDateOfBirth(): ?\DateTimeImmutable
{
return $this->dateOfBirth;
}
public function setDateOfBirth(\DateTimeImmutable $dateOfBirth): self
{
$this->dateOfBirth = $dateOfBirth;
return $this;
}
public function getIdentityVerified(): ?bool
{
return $this->identityVerified;
}
public function setIdentityVerified(bool $identityVerified): self
{
$this->identityVerified = $identityVerified;
return $this;
}
}And a User document with the following structure:
Sample document
__id: b83e319a-7887-11eb-8deb-b9e03d2e720d
username: daniel_johnson1
active: false
roles:
- CONTRIBUTOR
- AUTHOR
telephone: "+441254220959364"
password: "$2y$10$y8P2Cjph1F.iIc.s2j9aM.GW9qy8aOMeEfzDulQox465mgBJF.pPG"
person:
firstName: Daniel
lastName: Johnson
address:
house: '123'
street: Test Road
city: Testville
Country: Testshire
postcode: "TE1 3ST"
dateOfBirth: '1980-03-23T00:00:00+00:00'
identityVerified: trueWhen you initially retrieve the Document, the person key will contain an
array. But you can map this to your Person class as follows:
$user = $collection->get("b83e319a-7887-11eb-8deb-b9e03d2e720d");
$user->map('person', Person::class);
// $user->getPerson() now returns a Person object.
// Or you can map to an existing Person object.
$person = new Person();
$user->map('person', $person);Every document in the same collection must have a unique ID.
By default, when you create a new document, an ID is generated for you as a v1 UUID.
You can get or set a Document ID with the getId() and setId(string $id)
methods.
💡 Note: Changing a
DocumentID essentially treats it as a different document, i.e. providing a new unique ID will result in a new document being inserted in to your database when you save it. Likewise changing a document's ID to the ID of another document in the collection will cause that document to be overwritten.
If the ID was auto generated, you can obtain a DateTimeImmutable representing the
document's time of creation by calling its getTime() method:
$users = $db->collection("Users");
// Create a new Document with an auto generated UUID.
$user = $users->get();
// $date is a \DateTimeImmutable
$date = $user->getTime();
echo $date->format('d m Y H:i');If you don't want to use an auto-generated ID for a new document, simply pass in your
own ID to the collection's get() method. As long as the ID does not match any document
in the collection's database storage, a new document will be created. Document IDs are
strings.
$users = $db->collection("Users");
// Create a new Document with a custom ID.
// If this ID already exists in the Users collection, that document will be returned.
$user = $users->get("user_3815");Documents represented as a DocLite Document object provide a convenience method to
save the document to its collection. To save a Document in storage, call save().
$users = $db->collection("Users");
$user = $users->get();
$user->setUsername("admin");
$user->save();If a document has been mapped on to a custom class, you will need to save it through its collection instead.
$users = $db->collection("Users");
// Create a new document with an automatically generated UUID and
// retrieved as an object of type CustomUser.
$user = $users->get(null, CustomUser::class);
$user->setUsername("admin");
$users->save($user);Documents represented as a DocLite Document object provide a convenience method to
delete the document from its collection. To delete a Document in storage,
call delete().
$users = $db->collection("Users");
$user = $users->get("12345");
$user->delete();If a document has been mapped on to a custom class, you will need to delete it through its collection instead.
$users = $db->collection("Users");
// Create a new document with an automatically generated UUID and
// retrieved as an object of type CustomUser.
$user = $users->get("12345", CustomUser::class);
$users->deleteDocument($user);It is possible to add JSON Schema validation to a Document via the addJsonSchema()
method. This takes a single string parameter of a valid JSON schema. If the schema
cannot be validated, a DatabaseException will be thrown.
$user->addJsonSchema(file_get_contents('schema.json'));Once you have loaded a schema, every time you set a document property or try to save
the document, the document data will be validated against your schema. If the data fails
to validate, a DatabaseException will be thrown.
You can also manually validate at any time by calling validateJsonSchema().
$user->addJsonSchema(file_get_contents('schema.json'));
try {
$user->validateJsonSchema();
// This will automatically call validateJsonSchema() anyway.
$user->save();
// As will this.
$user->setUsername("foobar");
} catch (DatabaseException $e) {
$params = $e->getParams();
$error = $params['error'];
echo "Document failed to validate against JSON Schema because:\n".$error;
}Finally, you can unload a JSON Schema and remove the validaton by calling
removeJsonSchema().
$user->removeJsonSchema();Although there is not a specific integration with the Symfony framework, it's
trivial to inject DocLite as a service in to any Symfony application. Simply
install DocLite via Composer as an app dependency, then modify your
services.yaml as per the following example.
app.filedatabase:
class: Gebler\Doclite\FileDatabase
arguments:
$path: "../var/data/app.db"
$readOnly: false
app.memorydatabase:
class: Gebler\Doclite\MemoryDatabase
Gebler\Doclite\DatabaseInterface: '@app.filedatabase'
Gebler\Doclite\DatabaseInterface $memoryDb: '@app.memorydatabase'You can now typehint a DatabaseInterface like any other service, using the
alias $memoryDb as the parameter name if you'd like a MemoryDatabase.
DocLite is available under the MIT license as open source software.
If you use DocLite and find it useful, I am very grateful for any support towards its future development.
Please raise an issue on the project GitHub if you encounter any problems. I am always interested in improving the software.
You can email me on [email protected]
