GNU ELPA - triples

triples

Description
A flexible triple-based database for use in apps
Latest
triples-0.3.5.tar, 2023-Aug-10, 130 KiB
Maintainer
Andrew Hyatt <ahyatt@gmail.com>
Website
https://github.com/ahyatt/triples
Browse ELPA's repository
CGit or Gitweb
Badge

To install this package, run in Emacs:

M-x package-install RET triples RET

Full description

The triples module is a standard database module designed for use in other emacs modules. It works with either the builtin sqlite in Emacs 29 or the emacsql module, and provides a simple way of storing entities and their associated schema. The triples module is well suited to graph-like applications, where links between entities are important. The module has wrappers for most common operations, but it is anticipated that occasionally client modules would need to make their own sqlite calls. Many different database instances can be handled by the triples module. It is expected that clients supply the database connection. However, a standard triples database can be used, which is defined in triples-default-database-filename, and used when no filename is used to connect to by clients of the triples library.

1 Installing

This module is available through GNU ELPA, and can be installed as normal. However, most of the time this module is only useful in concert with another module which uses it as a library and will declare it as a dependency, so unless you are planning on developing with it, there is usually no need to install it directly.

2 Maturity

This module is somewhat new and should be considered beta quality.

While it has basic functionality, there are significant parts, such as a querying language, that are missing. Whether anyone needs such parts will determine the priority in which they get built.

3 Using the triples library

3.1 Types and Schema

triples employs a design in which each entity can be a member of many types, and each type has multiple properties. The properties that a type has is defined by schema. Let's take an example:

;; We assume a database called db has already been set up.
(triples-add-schema db 'person
       '(name :base/unique t :base/type string)
       '(age :base/unique t :base/type integer))
(triples-add-schema db 'employee
       '(id :base/unique t :base/type integer)
       '(manager :base/unique t)
       '(reportees :base/virtual-reversed employee/manager))

This adds a type called person, which can be set on any entity. There's another type called employee, which can also be set, independently of other types. This schema is stored in the database itself, so the database can function properly regardless on what elisp has been loaded. The schema can be redefined multiple times without any issues.

The person has 2 properties, name, and age. They are both marked as unique, so they take a single value, not a list. If :base/unique was not true, the value would be a list. We also specify what type it is, which can be any elisp type. employee is similarly constructed, but has an interesting property, reportees, which is a base/virtual-reversed property, meaning that it is supplied with values, but rather can get them from the reversed relation of employee/manager.

We'll explore how these types are used can be used in the section after next.

3.2 The triples concept

A triple is a unit of data consisting of a subject, a predicate, an object, and, optionally, internal metadata about the unit. The triple can be thought of as a link between the subject and object via the predicate.

Let's say that, as in the example above, we want to store someone's name. The triples would be a subject that uniquely identifies the person, a predicate that indicates the link between subject and object is about a name, and the object, which is the name value.

The object can become the subject, and this explains how the base/virtual-reversed predicate works. If Bob is the manager of Alice, then there could be a triple with Alice as the subject, manager as the predicate, and Bob as the object. But we can also find the reversed links, and ask who all are all the people that Bob manages. In this case, Bob is the subject, and Alice is the object. However, we don't actually need to store this information and try to keep it in sync, we can just get it by querying for when the Bob is the object and manager is the predicate.

3.3 Connecting

Before a database can be used, it should be connected with. This is done by the triples-connect function, which can be called with a filename or without. If a filename isn't given, a default one for the triples library, given in triples-default-database-filename is used. This provides a standard database for those that want to take advantage of the possibilities of having data from different sources that can build on each other.

An example of using this standard database is simply:

(let ((db (triples-connect)))
  (do-something-with db)
  (do-something-else-with db))

You could also use a global variable to hold the database connection, if you need the database to be active during many user actions.

3.4 Setting and retrieving

A subject can be set all at once (everything about the subject), or dealt with per-type. For example, the following are equivalent:

(triples-delete-subject db "alice")
(triples-set-type db "alice" 'person :name "Alice Aardvark" :age 41)
(triples-set-type db "alice" 'employee :id 1901 :manager "bob")
(triples-set-subject db "alice" '(person :name "Alice Aardvark" :age 41)
		     '(employee :id 1901 :manager "bob"))

In the second, the setting of the entire subject implies deleting everything previously associated with it.

Here is how the data is retrieved:

(triples-get-subject db "alice")

Which returns, assuming we have "catherine" and "dennis" who have "alice" as their employee/manager:

'(:person/name "Alice Aardvark" :person/age 41 :employee/id 1901 :employee/manager "bob" :employee/reportees '("catherine" "dennis"))

Or,

(triples-get-type db "alice" 'employee)

Which returns

'(:manager "bob" :reportees '("catherine" "dennis"))

There are other useful functions, including:

  • triples-get-types, which gets all the types a subject has,
  • triples-delete-subject, which deletes all data associated with a subject,
  • triples-with-predicate, gets all triples that is about a specific property,
  • triples-subject-with-predicate-object, get all subjects whose predicate is equal to object,
  • triples-subjects-of-type, get all subjects which have a particular type.

3.5 Predicates, with type and without

Sometimes the triples library will require predicates that are without type, and sometimes with type, or "combined predicates". The rule is that if the type is already specified in the function, it does not need to be respecified. If the type is not specified, it is included in the combined predicate.

When returning data, if data is from just one type, the type is not returned in the returned predicates. If the data is from multiple types, the type is returned as combined predicates.

3.6 Using direct SQL access

Sometimes clients of this library need to do something with the database, and the higher-level triples functionality doesn't help. If you would like lower-level functionality into handling triples, you can use the same low-level methods that the rest of this library uses. These start with triples-db-.

  • triples-db-insert: Add a triple. Uses SQL's REPLACE command, so there can't be completely duplicate triples (including the property, which often can serve as a disambiguation mechanism).
  • triples-db-delete: Delete triples matching the arguments. Empty arguments match everything, so (triples-db-delete db) will delete all triples.
  • triples-db-delete-subject-predicate-prefix: Delete triples matching subjects and with predicates with a certain prefix. This can't be done with triples-db-delete because that method uses exact matching for all arguments, and this uses prefix matching for the predicate.
  • triples-db-select-pred-op: Select triples that contain, for a predicate, an object with some relationship to the passed in value. This function lets you look for values equal to, greater, less, than or, "like", the passed in value.
  • triples-db-select: Select triples matching any of the parts of the triple. Like triples-db-delete, empty arguments match everything. You can specify exactly what to return with a selector.

Sometimes this still doesn't cover what you might want to do. In that case, you should write your own direct database access. However, please follow the coding patterns for the functions above in writing it, so that the code works with both Emacs 29's builtin sqlite, and emacsql.

3.7 Backups

If your application wants to back up your database, the function triples-backup provides the capability to do so safely. It can be called like:

(triples-backup db db-file 3)

Where db is the database, db-file is the filename where that database is stored, and 3 is the number of most recent backup files to keep. All older backup files will be deleted. The backup is stored where other emacs file backups are kept, defined by backup-directory-alist.

The triples-backups module provides a way to backup a database in a way defined in the database itself (so multiple clients of the same database can work in a sane way together). The number of backups to be kept, along with the "strategy" of when we want backups to happen is defined once per database.

;; Set up a backup configuration if none exists.
(require 'triples-backups)
(unless (triples-backups-configuration db)
  (triples-backups-setup db 3 'daily))

Once this is set up, whenever a change happens, simply call triples-backups-maybe-backup with the database and the filename where the database was opened from, which will back up the database if appropriate. This should be done after any important database write, once the action, at the application level, is finished. The triples module doesn't know when an appropriate point would be, so this is up to the client to run.

(defun my-package-add-data (data)
  (my-package-write-new-data package-db data)
  (triples-backups-maybe-backup db db-filename))

4 Using triples to develop apps with shared data

One possibility that arises from a design with entities (in triples terms, subjects) having multiple decomposable types like is done in the triples library is the possibility of many modules using the same database, each one adding their own data, but being able to make use out of each other's data.

For example, in the examples above we have a simple system for storing data about people and employees. If another module adds a type for annotations, now you can potentially annotate any entity, including people and employees. If another module adds functionality to store and complete on email addresses, now people, employees, and potentially types added by other modules such as organizations could have email addresses.

If this seems to fit your use case, you may want to try to just use the default database. The downside of this is that nothing prevents other modules from changing, corrupting or deleting your data.

Old versions

triples-0.3.4.tar.lz2023-Jul-2725.2 KiB
triples-0.3.3.tar.lz2023-Jun-2224.7 KiB
triples-0.3.2.tar.lz2023-Jun-1424.6 KiB
triples-0.3.1.tar.lz2023-Jun-1224.6 KiB
triples-0.3.tar.lz2023-Jun-1023.3 KiB
triples-0.2.7.tar.lz2023-Apr-1623.1 KiB
triples-0.2.6.tar.lz2023-Mar-0222.1 KiB
triples-0.2.5.tar.lz2023-Feb-2222.0 KiB
triples-0.2.3.tar.lz2023-Jan-1621.7 KiB
triples-0.2.2.tar.lz2023-Jan-0721.6 KiB
triples-0.2.1.tar.lz2023-Jan-0321.5 KiB
triples-0.2.tar.lz2023-Jan-0118.9 KiB
triples-0.1.2.tar.lz2022-Dec-1018.1 KiB
triples-0.1.1.tar.lz2022-Dec-0118.1 KiB
triples-0.1.tar.lz2022-Nov-0718.0 KiB

News

TITLE: Changelog for the triples module for GNU Emacs.

1 0.3.5

  • Compilation issues, and fixing an issue with not being able to use triples-with-transaction in some cases.

2 0.3.4

  • Fix instances where the database has no index, and has duplicate rows because of that index.
  • Fix differences in the properties column between emacsql and builtin when upgrading from emacsql to builtin.

3 0.3.3

  • Fix error in upgrade code SQL that occurs when integer conflicts are found.

4 0.3.2

  • Remove hard dependency on the sqlite library, which is a problem for emacs 28 users.

5 0.3.1

  • Fix issue with issue where duplicate values could interfere with table index creation during upgrade, causing type duplication.

6 0.3

  • All integers are stored as integers, and not strings. Applications using this library in previous versions should have users run triples-upgrade-to-0.3.
  • Fix for issue where adding schema would overwrite non-schema data on the same subject.

7 0.2.7

  • Add new function triples-db-select-pred-op, which allows querying among predicates for objects with a certain relationship to values, replaces triples-db-select-predicate-object-fragment.
  • Add ability to store cons types (basically lists) as values.

8 0.2.6

  • Fix bug where the functions triples-subjects-with-predicate-object could return the same subject multiple times.
  • Fix bug where backups were causing messages about "obsolete timestamp" for some users on Emacs 28.2.

9 0.2.5

  • Fix bug where backing up a nil filename resulted in an error.
  • Fix bug where strings are wrongly escapified, distoring text especially when repeatedly saved.

10 0.2.4

  • Move the CHANGELOG.org file to NEWS.org so the changes show up in GNU ELPA.

11 0.2.3

  • Allow nil for filename arguments in the backup functions. This will default to backing up the default database.
  • Fix issue with fallback for bad backup strategies.

12 0.2.2

  • Fix error behavior using Emacs builtin sqlite. Now error is rethrown instead of swallowed.

13 0.2.1

  • Add backup strategy never.

14 0.2

  • Create a default database to encourage a shared triple database. Add information on why this is an interesting idea in the README.
  • Add support for backups of databases via triples-backup, and a simple way to have a sane and shared backups created with the new triples-backups module.
  • Add triples-move-subject which will move both a subject as well as reference to it.

15 0.1.2

  • Bugfix release to remove backward compatibility with pre-Emacs 29 versions.

16 0.1.1

  • Bugfix release to fix triples-subject-with-predicate-object.

17 0.1

  • This is the initial version that contained basic triple functionality, and was integrated into GNU ELPA.