(feature): use sqlite as backing database (#200)

All org-roam related information will now be stored in the database. Henceforth, the cache needs to be built synchronously once (via `M-x org-roam-build-cache`), which is then incrementally updated.

CHANGELOG.md

@@ -2,6 +2,12 @@
 
 ## 0.1.3 (TBD)
 
+The biggest change, by far, the shift of database storage into SQLite.
+This means that the org-roam cache needs to be built manually at least
+once via `M-x org-roam-build-cache`.
+### Breaking Changes
+* [#200][gh-200] Move Org-roam cache into a SQLite database.
+
 ### New Features
 * [#182][gh-182] Support file name aliases via `#+ROAM_ALIAS`.
 * [#188][gh-188] Add `org-roam-protocol`, shifting `roam://` link handling into Emacs-lisp.
@@ -99,6 +105,7 @@ Mostly a documentation/cleanup release.
 [gh-165]: https://github.com/jethrokuan/org-roam/pull/165
 [gh-182]: https://github.com/jethrokuan/org-roam/pull/182
 [gh-188]: https://github.com/jethrokuan/org-roam/pull/188
+[gh-200]: https://github.com/jethrokuan/org-roam/pull/200
 
  # Local Variables:
  # eval: (auto-fill-mode -1)

README.md

@@ -28,10 +28,9 @@ As of February 2020, it is in a very early stage of development.
 
 Here's a screenshot of `org-roam`. The `org-roam` buffer shows
 backlinks for the active org buffer in the left window, as well as the
-surrounding content in the backlink file. The backlink database is
+surrounding content in the backlink file. The database is built once,
-built asynchronously in the background, and is not noticeable to the
+and updated incrementally. The graph is generated from the link
-end user. The graph is generated from the link structure, and can be
+structure, and can be used to navigate to the respective files.
-used to navigate to the respective files.
 
 ![img](doc/images/org-roam-graph.gif)
 

doc/configuration.md

@@ -35,9 +35,12 @@ Here is an example `.dir-locals.el` file that would be placed in a
 second Org-roam directory.
 
 ```emacs-lisp
-((nil . ((eval . (setq-local org-roam-directory (locate-dominating-file default-directory ".dir-locals.el"))))))
+((nil . ((org-roam-directory . "/path/to/here/"))))
 ```
 
+Remember to run `org-roam-build-cache` from a file within that
+directory, at least once.
+
 ## Org-roam Buffer
 
 The Org-roam buffer defaults to popping up from the right. You may

doc/tour.md

@@ -13,13 +13,11 @@ Without further ado, let's begin!
 
 ## Building the Cache
 
-Assuming you've set `org-roam-directory` appropriately, running `M-x
+The cache is a sqlite database named `org-roam.db`, which resides at
-org-roam--build-cache-async` should build up the caches that will
+the root of your `org-roam-directory`. To begin, we need to do a first
-allow you to begin using Org-roam. I do this on startup:
+build of this cache. To do so, run `M-x org-roam-build-cache`. This
-
+may take a while the first time, but is generally instantaneous in
-```emacs-lisp
+subsequent runs.
-(add-hook 'after-init-hook 'org-roam--build-cache-async)
-```
 
 ## Finding a Note
 

org-roam-db.el

@@ -0,0 +1,146 @@
+;;; org-roam-db.el --- Roam Research replica with Org-mode -*- coding: utf-8; lexical-binding: t -*-
+
+;; Copyright © 2020 Jethro Kuan <jethrokuan95@gmail.com>
+
+;; Author: Jethro Kuan <jethrokuan95@gmail.com>
+
+;; This file is NOT part of GNU Emacs.
+
+;; This program is free software; you can redistribute it and/or modify
+;; it under the terms of the GNU General Public License as published by
+;; the Free Software Foundation; either version 3, or (at your option)
+;; any later version.
+;;
+;; This program is distributed in the hope that it will be useful,
+;; but WITHOUT ANY WARRANTY; without even the implied warranty of
+;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+;; GNU General Public License for more details.
+;;
+;; You should have received a copy of the GNU General Public License
+;; along with GNU Emacs; see the file COPYING.  If not, write to the
+;; Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor,
+;; Boston, MA 02110-1301, USA.
+
+;;; Commentary:
+;;
+;; This code is heavily referenced from https://github.com/magit/forge.
+;;
+;;; Code:
+
+(require 'emacsql)
+(require 'emacsql-sqlite)
+
+;;; Options
+
+(defcustom org-roam-directory (expand-file-name "~/org-roam/")
+  "Default path to Org-roam files.
+
+All Org files, at any level of nesting, is considered part of the Org-roam."
+  :type 'directory
+  :group 'org-roam)
+
+(defconst org-roam-db-filename "org-roam.db"
+  "Name of the Org-roam database file.")
+
+(defconst org-roam--db-version 1)
+(defconst org-roam--sqlite-available-p
+  (with-demoted-errors "Org-roam initialization: %S"
+    (emacsql-sqlite-ensure-binary)
+    t))
+
+(defvar org-roam--db-connection (make-hash-table :test #'equal)
+  "Database connection to Org-roam database.")
+
+;;; Core
+
+(defun org-roam--get-db ()
+  "Return the sqlite db file."
+  (interactive "P")
+  (expand-file-name org-roam-db-filename org-roam-directory))
+
+(defun org-roam--get-db-connection ()
+  "Return the database connection, if any."
+  (gethash (file-truename org-roam-directory)
+           org-roam--db-connection))
+
+(defun org-roam-db ()
+  (unless (and (org-roam--get-db-connection)
+               (emacsql-live-p (org-roam--get-db-connection)))
+    (let* ((db-file (org-roam--get-db))
+           (init-db (not (file-exists-p db-file))))
+      (make-directory (file-name-directory db-file) t)
+      (let ((conn (emacsql-sqlite db-file)))
+        (puthash (file-truename org-roam-directory)
+                 conn
+                 org-roam--db-connection)
+        (when init-db
+          (org-roam--db-init conn))
+        (let* ((version (caar (emacsql conn "PRAGMA user_version")))
+               (version (org-roam--db-maybe-update conn version)))
+          (cond
+           ((> version org-roam--db-version)
+            (emacsql-close conn)
+            (user-error
+             "The Org-roam database was created with a newer Org-roam version. %s"
+             "You need to update the Org-roam package.")
+            ((< version org-roam--db-version)
+             (emacsql-close conn)
+             (error "BUG: The Org-roam database scheme changed %s"
+                    "and there is no upgrade path"))))))))
+  (org-roam--get-db-connection))
+
+;;; Api
+
+(defun org-roam-sql (sql &rest args)
+  (if  (stringp sql)
+      (emacsql (org-roam-db) (apply #'format sql args))
+    (apply #'emacsql (org-roam-db) sql args)))
+
+;;; Schemata
+
+(defconst org-roam--db-table-schemata
+  '((files
+     [(file :unique :primary-key)
+      (hash :not-null)
+      (last-modified :not-null)
+      ])
+
+    (file-links
+     [(file-from :not-null)
+      (file-to :not-null)
+      (properties :not-null)])
+
+    (titles
+     [
+      (file :not-null)
+      titles])
+
+    (refs
+     [(ref :unique :not-null)
+      (file :not-null)])))
+
+(defun org-roam--db-init (db)
+  (emacsql-with-transaction db
+    (pcase-dolist (`(,table . ,schema) org-roam--db-table-schemata)
+      (emacsql db [:create-table $i1 $S2] table schema))
+    (emacsql db (format "PRAGMA user_version = %s" org-roam--db-version))))
+
+(defun org-roam--db-maybe-update (db version)
+  (emacsql-with-transaction db
+    'ignore
+    ;; Do nothing now
+    version))
+
+(defun org-roam--db-close (&optional db)
+  (unless db
+    (setq db (org-roam--get-db-connection)))
+  (when (and db (emacsql-live-p db))
+    (emacsql-close db)))
+
+(defun org-roam--db-close-all ()
+  (dolist (conn (hash-table-values org-roam--db-connection))
+    (org-roam--db-close conn)))
+
+(provide 'org-roam-db)
+
+;;; org-roam-db.el ends here

org-roam-protocol.el

@@ -2,10 +2,6 @@
 
 ;; Copyright © 2020 Jethro Kuan <jethrokuan95@gmail.com>
 ;; Author: Jethro Kuan <jethrokuan95@gmail.com>
-;; URL: https://github.com/jethrokuan/org-roam
-;; Keywords: org-mode, roam, convenience
-;; Version: 0.1.2
-;; Package-Requires: ((emacs "26.1") (org "9.0"))
 
 ;; This file is NOT part of GNU Emacs.
 
@@ -71,8 +67,7 @@ If the function returns nil, the filename is removed from the
 list of filenames passed from emacsclient to the server. If the
 function returns a non-nil value, that value is passed to the
 server as filename."
-  (let ((the-protocol (concat (regexp-quote org-roam-protocol-the-protocol)
+  (let ((the-protocol (concat (regexp-quote org-roam-protocol-the-protocol) ":")))
-				                      ":")))
     (when (string-match the-protocol fname)
       (cadr (split-string fname the-protocol)))))
 

org-roam-utils.el

@@ -1,222 +0,0 @@
-;;; org-roam-utils.el --- Roam Research replica with Org-mode -*- coding: utf-8; lexical-binding: t -*-
-
-;; Copyright © 2020 Jethro Kuan <jethrokuan95@gmail.com>
-
-;; Author: Jethro Kuan <jethrokuan95@gmail.com>
-;; URL: https://github.com/jethrokuan/org-roam
-;; Keywords: org-mode, roam, convenience
-;; Version: 0.1.2
-;; Package-Requires: ((emacs "26.1") (org "9.0"))
-
-;; This file is NOT part of GNU Emacs.
-
-;; This program is free software; you can redistribute it and/or modify
-;; it under the terms of the GNU General Public License as published by
-;; the Free Software Foundation; either version 3, or (at your option)
-;; any later version.
-;;
-;; This program is distributed in the hope that it will be useful,
-;; but WITHOUT ANY WARRANTY; without even the implied warranty of
-;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-;; GNU General Public License for more details.
-;;
-;; You should have received a copy of the GNU General Public License
-;; along with GNU Emacs; see the file COPYING.  If not, write to the
-;; Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor,
-;; Boston, MA 02110-1301, USA.
-
-;;; Commentary:
-;;
-;; This library is an attempt at injecting Roam functionality into Org-mode.
-;; This is achieved primarily through building caches for forward links,
-;; backward links, and file titles.
-;;
-;;; Code:
-
-(require 'org)
-(require 'org-element)
-(require 'ob-core) ;for org-babel-parse-header-arguments
-(require 'subr-x)
-(require 'cl-lib)
-
-(defun org-roam--file-name-extension (filename)
-  "Return file name extension for FILENAME.
-
-Like file-name-extension, but does not strip version number."
-  (save-match-data
-    (let ((file (file-name-nondirectory filename)))
-      (if (and (string-match "\\.[^.]*\\'" file)
-                     (not (eq 0 (match-beginning 0))))
-          (substring file (+ (match-beginning 0) 1))))))
-
-(defun org-roam--org-file-p (path)
-  "Check if PATH is pointing to an org file."
-  (let ((ext (org-roam--file-name-extension path)))
-    (or (string= ext "org")
-        (and
-         (string= ext "gpg")
-         (string= (org-roam--file-name-extension (file-name-sans-extension path)) "org")))))
-
-(defun org-roam--find-files (dir)
-  "Return all `org-roam' files in `DIR'."
-  (if (file-exists-p dir)
-      (let ((files (directory-files dir t "." t))
-            (dir-ignore-regexp (concat "\\(?:"
-                                       "\\."
-                                       "\\|\\.\\."
-                                       "\\)$"))
-            result)
-        (dolist (file files)
-          (cond
-           ((file-directory-p file)
-            (when (not (string-match dir-ignore-regexp file))
-              (setq result (append (org-roam--find-files file) result))))
-           ((and (file-readable-p file)
-                 (org-roam--org-file-p file))
-            (setq result (cons (file-truename file) result)))))
-        result)))
-
-(defun org-roam--parse-content (&optional file-path)
-  "Parse the current buffer, and return a list of items for processing."
-  (org-element-map (org-element-parse-buffer) 'link
-    (lambda (link)
-      (let ((type (org-element-property :type link))
-            (path (org-element-property :path link))
-            (start (org-element-property :begin link)))
-        (when (and (string= type "file")
-                   (org-roam--org-file-p path))
-          (goto-char start)
-          (let* ((element (org-element-at-point))
-                 (begin (or (org-element-property :content-begin element)
-                            (org-element-property :begin element)))
-                 (content (or (org-element-property :raw-value element)
-                              (buffer-substring
-                               begin
-                               (or (org-element-property :content-end element)
-                                   (org-element-property :end element)))))
-                 (content (string-trim content))
-                 (file-path (or file-path
-                                (file-truename (buffer-file-name (current-buffer))))))
-            (list :from file-path
-                  :to (file-truename (expand-file-name path (file-name-directory file-path)))
-                  :properties (list :content content :point begin))))))))
-
-(cl-defun org-roam--insert-item (item &key forward backward)
-  "Insert ITEM into FORWARD and BACKWARD cache.
-
-ITEM is of the form: (:from from-path :to to-path :properties (:content preview-content :point point))."
-  (pcase-let ((`(:from ,p-from :to ,p-to :properties ,props) item))
-    ;; Build forward-links
-    (let ((links (gethash p-from forward)))
-      (if links
-          (puthash p-from
-                   (if (member p-to links)
-                       links
-                     (cons p-to links)) forward)
-        (puthash p-from (list p-to) forward)))
-    ;; Build backward-links
-    (let ((contents-hash (gethash p-to backward)))
-      (if contents-hash
-          (if-let ((contents-list (gethash p-from contents-hash)))
-              (let ((updated (cons props contents-list)))
-                (puthash p-from updated contents-hash)
-                (puthash p-to contents-hash backward))
-            (progn
-              (puthash p-from (list props) contents-hash)
-              (puthash p-to contents-hash backward)))
-        (let ((contents-hash (make-hash-table :test #'equal)))
-          (puthash p-from (list props) contents-hash)
-          (puthash p-to contents-hash backward))))))
-
-(defun org-roam--extract-global-props (props)
-  "Extract PROPS from the current buffer."
-  (let ((buf (org-element-parse-buffer))
-        (res '()))
-    (dolist (prop props)
-      (let ((p (org-element-map
-                   buf
-                   'keyword
-                 (lambda (kw)
-                   (when (string= (org-element-property :key kw) prop)
-                     (org-element-property :value kw)))
-                 :first-match t)))
-        (setq res (cons (cons prop p) res))))
-    res))
-
-(defun org-roam--aliases-str-to-list (str)
-  "Function to transform string STR into list of alias titles.
-
-This snippet is obtained from ox-hugo:
-https://github.com/kaushalmodi/ox-hugo/blob/a80b250987bc770600c424a10b3bca6ff7282e3c/ox-hugo.el#L3131"
-  (when (stringp str)
-    (let* ((str (org-trim str))
-           (str-list (split-string str "\n"))
-           ret)
-      (dolist (str-elem str-list)
-        (let* ((format-str ":dummy '(%s)") ;The :dummy key is discarded in the `lst' var below.
-               (alist (org-babel-parse-header-arguments (format format-str str-elem)))
-               (lst (cdr (car alist)))
-               (str-list2 (mapcar (lambda (elem)
-                                    (cond
-                                     ((symbolp elem)
-                                      (symbol-name elem))
-                                     (t
-                                      elem)))
-                                  lst)))
-          (setq ret (append ret str-list2))))
-      ret)))
-
-(defun org-roam--extract-titles ()
-  "Extract the titles from current buffer.
-
-Titles are obtained via the #+TITLE property, or aliases
-specified via the #+ROAM_ALIAS property."
-  (let* ((props (org-roam--extract-global-props '("TITLE" "ROAM_ALIAS")))
-         (aliases (cdr (assoc "ROAM_ALIAS" props)))
-         (title (cdr (assoc "TITLE" props)))
-         (alias-list (org-roam--aliases-str-to-list aliases)))
-    (if title
-        (cons title alias-list)
-      alias-list)))
-
-(defun org-roam--extract-ref ()
-  "Extract the ref from current buffer."
-  (cdr (assoc "ROAM_KEY" (org-roam--extract-global-props '("ROAM_KEY")))))
-
-(defun org-roam--build-cache (dir)
-  "Build the org-roam caches in DIR."
-  (let ((backward-links (make-hash-table :test #'equal))
-        (forward-links (make-hash-table :test #'equal))
-        (file-titles (make-hash-table :test #'equal))
-        (refs (make-hash-table :test #'equal)))
-    (let* ((org-roam-files (org-roam--find-files dir))
-           (file-items (mapcar (lambda (file)
-                                 (with-temp-buffer
-                                   (insert-file-contents file)
-                                   (org-roam--parse-content file))) org-roam-files)))
-      (dolist (items file-items)
-        (dolist (item items)
-          (org-roam--insert-item
-           item
-           :forward forward-links
-           :backward backward-links)))
-      (dolist (file org-roam-files)
-        (with-temp-buffer
-          (insert-file-contents file)
-          (when-let ((titles (org-roam--extract-titles)))
-            (puthash file titles file-titles))
-          (when-let ((ref (org-roam--extract-ref)))
-            ;; FIXME: this overrides previous refs, should probably have a
-            ;; warning when ref is not unique
-            (puthash ref file refs)))
-        org-roam-files))
-    (list
-     :directory dir
-     :forward forward-links
-     :backward backward-links
-     :titles file-titles
-     :refs refs)))
-
-(provide 'org-roam-utils)
-
-;;; org-roam-utils.el ends here

org-roam.el

@@ -6,7 +6,7 @@
 ;; URL: https://github.com/jethrokuan/org-roam
 ;; Keywords: org-mode, roam, convenience
 ;; Version: 0.1.2
-;; Package-Requires: ((emacs "26.1") (dash "2.13") (f "0.17.2") (s "1.12.0") (async "1.9.4") (org "9.0"))
+;; Package-Requires: ((emacs "26.1") (dash "2.13") (f "0.17.2") (s "1.12.0") (org "9.0") (emacsql "3.0.0") (emacsql-sqlite "1.0.0"))
 
 ;; This file is NOT part of GNU Emacs.
 
@@ -35,14 +35,15 @@
 ;;; Code:
 
 (eval-when-compile (require 'cl-lib))
-(require 'dash)
+(require 'org)
 (require 'org-element)
-(require 'async)
+(require 'ob-core) ;for org-babel-parse-header-arguments
 (require 'subr-x)
+(require 'dash)
 (require 's)
 (require 'f)
-(require 'org-roam-utils)
+(require 'cl-lib)
-(require 'eieio)
+(require 'org-roam-db)
 
 ;;; Customizations
 (defgroup org-roam nil
@@ -52,12 +53,10 @@
   :link '(url-link :tag "Github" "https://github.com/jethrokuan/org-roam")
   :link '(url-link :tag "Online Manual" "https://org-roam.readthedocs.io/"))
 
-(defcustom org-roam-directory (expand-file-name "~/org-roam/")
+(defgroup org-roam-faces nil
-  "Path to Org-roam files.
+  "Faces used by Org-Roam."
-
+  :group 'org-roam
-All Org files, at any level of nesting, is considered part of the Org-roam."
+  :group 'faces)
-  :type 'directory
-  :group 'org-roam)
 
 (defcustom org-roam-new-file-directory nil
   "Path to where new Org-roam files are created.
@@ -66,11 +65,6 @@ If nil, default to the org-roam-directory (preferred)."
   :type 'directory
   :group 'org-roam)
 
-(defcustom org-roam-mute-cache-build nil
-  "Whether to mute the cache build message."
-  :type 'boolean
-  :group 'org-roam)
-
 (defcustom org-roam-buffer-position 'right
   "Position of `org-roam' buffer.
 
@@ -81,14 +75,6 @@ Valid values are
                  (const right))
   :group 'org-roam)
 
-(defcustom org-roam-file-name-function #'org-roam--file-name-timestamp-title
-  "The function used to generate filenames.
-
-The function takes as parameter `TITLE', a string the user inputs."
-  :group 'org-roam
-  :type '(choice (const :tag "Default" org-roam--file-name-timestamp-title)
-                 (function :tag "Personalized function")))
-
 (defcustom org-roam-link-title-format "%s"
   "The format string used when inserting org-roam links that use their title."
   :type 'string
@@ -126,55 +112,15 @@ If nil, always ask for filename."
   :group 'org-roam)
 
 (defcustom org-roam-graph-max-title-length 100
-  "Maximum length of titles in graphviz graph nodes"
+  "Maximum length of titles in Graphviz graph nodes."
   :type 'number
   :group 'org-roam)
 
 (defcustom org-roam-graph-node-shape "ellipse"
-  "Maximum length of titles in graphviz graph nodes"
+  "Shape of Graphviz nodes."
   :type 'string
   :group 'org-roam)
 
-(defgroup org-roam-faces nil
-  "Faces used by Org-Roam."
-  :group 'org-roam
-  :group 'faces)
-
-;;; Polyfills
-;; These are for functions I use that are only available in newer Emacs
-
-;; Introduced in Emacs 27.1
-(unless (fboundp 'make-empty-file)
-  (defun make-empty-file (filename &optional parents)
-    "Create an empty file FILENAME.
-Optional arg PARENTS, if non-nil then creates parent dirs as needed.
-
-If called interactively, then PARENTS is non-nil."
-    (interactive
-     (let ((filename (read-file-name "Create empty file: ")))
-       (list filename t)))
-    (when (and (file-exists-p filename) (null parents))
-      (signal 'file-already-exists `("File exists" ,filename)))
-    (let ((paren-dir (file-name-directory filename)))
-      (when (and paren-dir (not (file-exists-p paren-dir)))
-        (make-directory paren-dir parents)))
-    (write-region "" nil filename nil 0)))
-
-;;; Classes
-
-(defclass org-roam-cache ()
-  ((initialized :initarg :initialized
-                :documentation "Is cache valid?")
-   (forward-links :initarg :forward-links
-                  :documentation "Cache containing forward links.")
-   (backward-links :initarg :backward-links
-                   :documentation "Cache containing backward links.")
-   (titles :initarg :titles
-           :documentation "Cache containing titles for org-roam files.")
-   (refs :initarg :refs
-         :documentation "Cache with ref as key, and file path as value."))
-  "All cache for an org-roam directory.")
-
 ;;; Dynamic variables
 (defvar org-roam--current-buffer nil
   "Currently displayed file in `org-roam' buffer.")
@@ -182,70 +128,196 @@ If called interactively, then PARENTS is non-nil."
 (defvar org-roam-last-window nil
   "Last window `org-roam' was called from.")
 
-(defvar org-roam--cache (make-hash-table :test 'equal)
-  "The list of cache separated by directory.")
-
-(defvar-local org-roam--local-cache-ref nil
-  "Local reference of the buffer's cache object.")
-
-(defvar-local org-roam--local-cache-id nil
-  "Local reference of the buffer's cache object id, which is
-  comparable by \"eq\".")
-
 ;;; Utilities
-(defun org-roam-directory-normalized ()
+(defun org-roam--touch-file (path)
-  "Get the org-roam-directory normalized so that it can be used
+  "Touches an empty file at PATH."
-as a unique key."
+  (make-directory (file-name-directory path) t)
-  (directory-file-name (file-truename org-roam-directory)))
+  (f-touch path))
 
-(defmacro org-roam--get-local (name)
+(defun org-roam--file-name-extension (filename)
-  "Get a variable that is local to the current org-roam-directory."
+  "Return file name extension for FILENAME.
-  `(gethash (org-roam-directory-normalized) ,name nil))
+Like file-name-extension, but does not strip version number."
+  (save-match-data
+    (let ((file (file-name-nondirectory filename)))
+      (if (and (string-match "\\.[^.]*\\'" file)
+               (not (eq 0 (match-beginning 0))))
+          (substring file (+ (match-beginning 0) 1))))))
 
-(defmacro org-roam--set-local (name value)
+(defun org-roam--org-file-p (path)
-  "Set a variable that is local to the current org-roam-directory."
+  "Check if PATH is pointing to an org file."
-  `(puthash (org-roam-directory-normalized) ,value ,name))
+  (let ((ext (org-roam--file-name-extension path)))
+    (or (string= ext "org")
+        (and
+         (string= ext "gpg")
+         (string= (org-roam--file-name-extension (file-name-sans-extension path)) "org")))))
 
-(defun org-roam--get-directory-cache ()
+(defun org-roam--find-files (dir)
-  "Get the cache object for the current org-roam-directory."
+  "Return all `org-roam' files in `DIR'."
-  (unless (eq org-roam--local-cache-id org-roam-directory)
+  (if (file-exists-p dir)
-    ;; Prevent needless repeated calls to org-roam-directory-normalized by
+      (let ((files (directory-files dir t "." t))
-    ;; having a reference to the cache object that matches the local buffer.
+            (dir-ignore-regexp (concat "\\(?:"
-    (setq org-roam--local-cache-ref
+                                       "\\."
-          (let* ((cache (org-roam--get-local org-roam--cache)))
+                                       "\\|\\.\\."
-            (if cache
+                                       "\\)$"))
-                cache
+            result)
-              (let ((new-cache (org-roam--default-cache)))
+        (dolist (file files)
-                (org-roam--set-local org-roam--cache new-cache)
+          (cond
-                new-cache))))
+           ((file-directory-p file)
-    (setq org-roam--local-cache-id org-roam-directory))
+            (when (not (string-match dir-ignore-regexp file))
-  org-roam--local-cache-ref)
+              (setq result (append (org-roam--find-files file) result))))
+           ((and (file-readable-p file)
+                 (org-roam--org-file-p file))
+            (setq result (cons (file-truename file) result)))))
+        result)))
+
+(defun org-roam--get-links (&optional file-path)
+  "Get the links in the buffer.
+If FILE-PATH is passed, use that as the source file."
+  (let ((file-path (or file-path
+                       (file-truename (buffer-file-name (current-buffer))))))
+    (org-element-map (org-element-parse-buffer) 'link
+      (lambda (link)
+        (let ((type (org-element-property :type link))
+              (path (org-element-property :path link))
+              (start (org-element-property :begin link)))
+          (when (and (string= type "file")
+                     (org-roam--org-file-p path))
+            (goto-char start)
+            (let* ((element (org-element-at-point))
+                   (begin (or (org-element-property :content-begin element)
+                              (org-element-property :begin element)))
+                   (content (or (org-element-property :raw-value element)
+                                (buffer-substring
+                                 begin
+                                 (or (org-element-property :content-end element)
+                                     (org-element-property :end element)))))
+                   (content (string-trim content)))
+              (vector file-path
+                      (file-truename (expand-file-name path (file-name-directory file-path)))
+                      (list :content content :point begin)))))))))
+
+(defun org-roam--extract-global-props (props)
+  "Extract PROPS from the current buffer."
+  (let ((buf (org-element-parse-buffer))
+        (res '()))
+    (dolist (prop props)
+      (let ((p (org-element-map
+                   buf
+                   'keyword
+                 (lambda (kw)
+                   (when (string= (org-element-property :key kw) prop)
+                     (org-element-property :value kw)))
+                 :first-match t)))
+        (setq res (cons (cons prop p) res))))
+    res))
+
+(defun org-roam--aliases-str-to-list (str)
+  "Function to transform string STR into list of alias titles.
+
+This snippet is obtained from ox-hugo:
+https://github.com/kaushalmodi/ox-hugo/blob/a80b250987bc770600c424a10b3bca6ff7282e3c/ox-hugo.el#L3131"
+  (when (stringp str)
+    (let* ((str (org-trim str))
+           (str-list (split-string str "\n"))
+           ret)
+      (dolist (str-elem str-list)
+        (let* ((format-str ":dummy '(%s)") ;The :dummy key is discarded in the `lst' var below.
+               (alist (org-babel-parse-header-arguments (format format-str str-elem)))
+               (lst (cdr (car alist)))
+               (str-list2 (mapcar (lambda (elem)
+                                    (cond
+                                     ((symbolp elem)
+                                      (symbol-name elem))
+                                     (t
+                                      elem)))
+                                  lst)))
+          (setq ret (append ret str-list2))))
+      ret)))
+
+(defun org-roam--extract-titles ()
+  "Extract the titles from current buffer.
+Titles are obtained via the #+TITLE property, or aliases
+specified via the #+ROAM_ALIAS property."
+  (let* ((props (org-roam--extract-global-props '("TITLE" "ROAM_ALIAS")))
+         (aliases (cdr (assoc "ROAM_ALIAS" props)))
+         (title (cdr (assoc "TITLE" props)))
+         (alias-list (org-roam--aliases-str-to-list aliases)))
+    (if title
+        (cons title alias-list)
+      alias-list)))
+
+(defun org-roam--extract-ref ()
+  "Extract the ref from current buffer."
+  (cdr (assoc "ROAM_KEY" (org-roam--extract-global-props '("ROAM_KEY")))))
+
+(defun org-roam--insert-links (links)
+  "Insert LINK into the org-roam cache."
+  (org-roam-sql
+   [:insert :into file-links
+            :values $v1]
+   links))
+
+(defun org-roam--insert-titles (file titles)
+  "Insert TITLES into the org-roam-cache."
+  (org-roam-sql
+   [:insert :into titles
+            :values $v1]
+   (list (vector file titles))))
+
+(defun org-roam--insert-ref (file ref)
+  "Insert REF into the Org-roam cache."
+  (org-roam-sql
+   [:insert :into refs
+            :values $v1]
+   (list (vector ref file))))
+
+(defun org-roam--clear-cache ()
+  "Clears all entries in the caches."
+  (interactive)
+  (when (file-exists-p (org-roam--get-db))
+    (org-roam-sql [:delete :from files])
+    (org-roam-sql [:delete :from titles])
+    (org-roam-sql [:delete :from file-links])
+    (org-roam-sql [:delete :from files])
+    (org-roam-sql [:delete :from refs])))
+
+(defun org-roam--clear-file-from-cache (&optional filepath)
+  "Remove any related links to the file at FILEPATH.
+This is equivalent to removing the node from the graph."
+  (let* ((path (or filepath
+                   (buffer-file-name (current-buffer))))
+         (file (file-truename path)))
+    (org-roam-sql [:delete :from files
+                           :where (= file $s1)]
+                  file)
+    (org-roam-sql [:delete :from file-links
+                           :where (= file-from $s1)]
+                  file)
+    (org-roam-sql [:delete :from titles
+                           :where (= file $s1)]
+                  file)
+    (org-roam-sql [:delete :from refs
+                           :where (= file $s1)]
+                  file)))
+
+(defun org-roam--get-current-files ()
+  "Return a hash of file to buffer string hash."
+  (let* ((current-files (org-roam-sql [:select * :from files]))
+         (ht (make-hash-table :test #'equal)))
+    (dolist (row current-files)
+      (puthash (car row) (cadr row) ht))
+    ht))
 
 (defun org-roam--cache-initialized-p ()
-  "Is cache valid?"
+  "Whether the cache has been initialized."
-  (oref (org-roam--get-directory-cache) initialized))
+  (and (file-exists-p (org-roam--get-db))
-
+       (> (caar (org-roam-sql [:select (funcall count) :from titles]))
-(defun org-roam--forward-links-cache ()
+          0)))
-  "Cache containing forward links."
-  (oref (org-roam--get-directory-cache) forward-links))
-
-(defun org-roam--backward-links-cache ()
-  "Cache containing backward links."
-  (oref (org-roam--get-directory-cache) backward-links))
-
-(defun org-roam--titles-cache ()
-  "Cache containing titles for org-roam files."
-  (oref (org-roam--get-directory-cache) titles))
-
-(defun org-roam--refs-cache ()
-  "Cache containing refs for org-roam files."
-  (oref (org-roam--get-directory-cache) refs))
 
 (defun org-roam--ensure-cache-built ()
   "Ensures that org-roam cache is built."
   (unless (org-roam--cache-initialized-p)
-    (org-roam--build-cache-async)
+    (error "[Org-roam] your cache isn't built yet! Please wait.")))
-    (user-error "Your Org-Roam cache isn't built yet! Please wait")))
 
 (defun org-roam--org-roam-file-p (&optional file)
   "Return t if FILE is part of org-roam system, defaulting to the name of the current buffer. Else, return nil."
@@ -258,11 +330,10 @@ as a unique key."
 
 (defun org-roam--get-titles-from-cache (file)
   "Return titles and aliases of `FILE' from the cache."
-  (or (gethash file (org-roam--titles-cache))
+  (caar (org-roam-sql [:select [titles] :from titles
-      (progn
+                       :where (= file $s1)]
-        (unless (org-roam--cache-initialized-p)
+                      file
-          (user-error "The Org-Roam caches aren't built! Please run org-roam--build-cache-async"))
+                      :limit 1)))
-        nil)))
 
 (defun org-roam--get-title-from-cache (file)
   "Return the title of `FILE' from the cache."
@@ -341,7 +412,7 @@ It uses TITLE and the current timestamp to form a unique title."
       (setq file-path (org-roam--new-file-path (file-name-fn title) t))
       (if (file-exists-p file-path)
           file-path
-        (make-empty-file file-path t)
+        (org-roam--touch-file file-path)
         (write-region
          (s-format (plist-get template :content)
                    'aget
@@ -350,18 +421,6 @@ It uses TITLE and the current timestamp to form a unique title."
          nil file-path nil)
         file-path))))
 
-(defun org-roam--get-new-id (title)
-  "Return a new ID, given the note TITLE."
-  (let* ((proposed-slug (funcall org-roam-file-name-function title))
-         (new-slug (if org-roam-filename-noconfirm
-                       proposed-slug
-                     (read-string "Enter ID (without extension): "
-                                  proposed-slug)))
-         (file-path (org-roam--new-file-path new-slug t)))
-    (if (file-exists-p file-path)
-        (user-error "There's already a file at %s")
-      new-slug)))
-
 ;;; Inserting org-roam links
 (defun org-roam-insert (prefix)
   "Find an org-roam file, and insert a relative org link to it at point.
@@ -397,13 +456,16 @@ If PREFIX, downcase the title before insertion."
 ;;; Finding org-roam files
 (defun org-roam--get-title-path-completions ()
   "Return a list of cons pairs for titles to absolute path of Org-roam files."
-  (let ((files (org-roam--find-all-files))
+  (let* ((rows (org-roam-sql [:select [file titles] :from titles]))
-        (res '()))
+         res)
-    (dolist (file files)
+    (dolist (row rows)
-      (if-let (titles (org-roam--get-titles-from-cache file))
+      (let ((file-path (car row))
-          (dolist (title titles)
+            (titles (cadr row)))
-            (setq res (cons (cons title file) res)))
+        (if titles
-        (setq res (cons (cons (org-roam--path-to-slug file) file) res))))
+            (dolist (title titles)
+              (setq res (cons (cons title file-path) res)))
+          (setq res (cons (cons (org-roam--path-to-slug file-path)
+                                file-path) res)))))
     res))
 
 (defun org-roam-find-file ()
@@ -437,115 +499,102 @@ If PREFIX, downcase the title before insertion."
     (when-let ((name (completing-read "Choose a buffer: " names-and-buffers)))
       (switch-to-buffer (cdr (assoc name names-and-buffers))))))
 
-(defvar org-roam--ongoing-async-build (make-hash-table :test 'equal)
-  "Prevent multiple async cache builds.  This can happen when
-  restoring a session or loading multiple org-roam files before a
-  build has completed.")
-
 ;;; Building the org-roam cache
-(defun org-roam--build-cache-async (&optional on-success)
+(defun org-roam-build-cache ()
-  "Builds the caches asychronously."
+  "Build the cache for `org-roam-directory'."
   (interactive)
-  (let ((existing (org-roam--get-local org-roam--ongoing-async-build)))
+  (org-roam-db) ;; To initialize the database, no-op if already initialized
-    (unless (and (processp existing)
+  (let* ((org-roam-files (org-roam--find-files org-roam-directory))
-                 (not (async-ready existing)))
+         (current-files (org-roam--get-current-files))
-      (org-roam--set-local
+         (time (current-time))
-       org-roam--ongoing-async-build
+         all-files all-links all-titles all-refs)
-       (async-start
+    (dolist (file org-roam-files)
-        `(lambda ()
+      (with-temp-buffer
-           (setq load-path ',load-path)
+        (insert-file-contents file)
-           (package-initialize)
+        (let ((contents-hash (secure-hash 'sha1 (current-buffer))))
-           (require 'org-roam-utils)
+          (unless (string= (gethash file current-files)
-           ,(async-inject-variables "org-roam-directory")
+                           contents-hash)
-           (org-roam--build-cache org-roam-directory))
+            (org-roam--clear-file-from-cache file)
-        (lambda (cache)
+            (setq all-files
-          (let ((org-roam-directory (plist-get cache :directory)))
+                  (cons (vector file contents-hash time) all-files))
-            (let ((obj (org-roam--get-directory-cache)))
+            (when-let (links (org-roam--get-links file))
-              (oset obj initialized t)
+              (setq all-links (append links all-links)))
-              (oset obj forward-links (plist-get cache :forward))
+            (let ((titles (org-roam--extract-titles)))
-              (oset obj backward-links (plist-get cache :backward))
+              (setq all-titles (cons (vector file titles) all-titles)))
-              (oset obj titles (plist-get cache :titles))
+            (when-let ((ref (org-roam--extract-ref)))
-              (oset obj refs (plist-get cache :refs)))
+              (setq all-refs (cons (vector ref file) all-refs))))
-            (unless org-roam-mute-cache-build
+          (remhash file current-files))))
-              (message "Org-roam cache built!"))
+    (dolist (file (hash-table-keys current-files))
-            (when on-success
+      ;; These files are no longer around, remove from cache...
-              (funcall on-success)))))))))
+      (org-roam--clear-file-from-cache file))
-
+    (when all-files
-(defun org-roam--clear-cache ()
+      (org-roam-sql
-  "Clears all entries in the caches."
+       [:insert :into files
-  (interactive)
+                :values $v1]
-  (let ((cache (org-roam--get-directory-cache)))
+       all-files))
-    (oset cache initialized nil)
+    (when all-links
-    (oset cache forward-links (make-hash-table :test #'equal))
+      (org-roam-sql
-    (oset cache backward-links (make-hash-table :test #'equal))
+       [:insert :into file-links
-    (oset cache titles (make-hash-table :test #'equal))
+                :values $v1]
-    (oset cache refs (make-hash-table :test #'equal))))
+       all-links))
-
+    (when all-titles
-(defun org-roam--default-cache ()
+      (org-roam-sql
-  "A default, uninitialized cache object."
+       [:insert :into titles
-  (org-roam-cache :initialized nil
+                :values $v1]
-                  :forward-links (make-hash-table :test #'equal)
+       all-titles))
-                  :backward-links (make-hash-table :test #'equal)
+    (when all-refs
-                  :titles (make-hash-table :test #'equal)
+      (org-roam-sql
-                  :refs (make-hash-table :test #'equal)))
+       [:insert :into refs
-
+                :values $v1]
-(defun org-roam--clear-file-from-cache (&optional filepath)
+       all-refs))
-  "Remove any related links to the file at FILEPATH.
+    (let ((stats (list :files (length all-files)
-This is equivalent to removing the node from the graph."
+                       :links (length all-links)
-  (let* ((path (or filepath
+                       :titles (length all-titles)
-                   (buffer-file-name (current-buffer))))
+                       :refs (length all-refs)
-         (file (file-truename path)))
+                       :deleted (length (hash-table-keys current-files)))))
-    ;; Step 1: Remove all existing links for file
+      (message (format "files: %s, links: %s, titles: %s, refs: %s, deleted: %s"
-    (when-let ((forward-links (gethash file (org-roam--forward-links-cache))))
+                       (plist-get stats :files)
-      ;; Delete backlinks to file
+                       (plist-get stats :links)
-      (dolist (link forward-links)
+                       (plist-get stats :titles)
-        (when-let ((backward-links (gethash link (org-roam--backward-links-cache))))
+                       (plist-get stats :refs)
-          (remhash file backward-links)
+                       (plist-get stats :deleted)))
-          (puthash link backward-links (org-roam--backward-links-cache))))
+      stats)))
-      ;; Clean out forward links
-      (remhash file (org-roam--forward-links-cache)))
-    ;; Step 2: Remove from the title cache
-    (remhash file (org-roam--titles-cache))
-    ;; Step 3: Remove from the refs cache
-    (maphash (lambda (k v)
-               (when (string= v file)
-                 (remhash k (org-roam--refs-cache))))
-             (org-roam--refs-cache))))
 
 (defun org-roam--update-cache-titles ()
-  "Insert the title of the current buffer into the cache."
+  "Update the title of the current buffer into the cache."
-  (when-let ((titles (org-roam--extract-titles)))
+  (let ((file (file-truename (buffer-file-name (current-buffer)))))
-    (puthash (file-truename (buffer-file-name (current-buffer)))
+    (org-roam-sql [:delete :from titles
-             titles
+                           :where (= file $s1)]
-             (org-roam--titles-cache))))
+                  file)
+    (org-roam--insert-titles file (org-roam--extract-titles))))
 
 (defun org-roam--update-cache-refs ()
-  "Insert the ref of the current buffer into the cache."
+  "Update the ref of the current buffer into the cache."
-  (when-let ((ref (org-roam--extract-ref)))
+  (let ((file (file-truename (buffer-file-name (current-buffer)))))
-    (puthash ref
+    (org-roam-sql [:delete :from refs
-             (file-truename (buffer-file-name (current-buffer)))
+                   :where (= file $s1)]
-             (org-roam--refs-cache))))
+                  file)
+    (when-let ((ref (org-roam--extract-ref)))
+      (org-roam--insert-ref file ref))))
+
+(defun org-roam--update-cache-links ()
+  "Update the file links of the current buffer in the cache."
+  (let ((file (file-truename (buffer-file-name (current-buffer)))))
+    (org-roam-sql [:delete :from file-links
+                   :where (= file-from $s1)]
+                  file)
+    (when-let ((links (org-roam--get-links)))
+      (org-roam--insert-links links))))
 
 (defun org-roam--update-cache ()
   "Update org-roam caches for the current buffer file."
   (save-excursion
-    (org-roam--clear-file-from-cache)
-    ;; Insert into title cache
     (org-roam--update-cache-titles)
-    ;; Insert into ref cache
     (org-roam--update-cache-refs)
-    ;; Insert new items
+    (org-roam--update-cache-links)
-    (let ((items (org-roam--parse-content)))
-      (dolist (item items)
-        (org-roam--insert-item
-         item
-         :forward (org-roam--forward-links-cache)
-         :backward (org-roam--backward-links-cache))))
-    ;; Rerender buffer
     (org-roam--maybe-update-buffer :redisplay t)))
 
 ;;; Org-roam daily notes
-
 (defun org-roam--file-for-time (time)
   "Create and find file for TIME."
   (let* ((org-roam-templates (list (list "daily" (list :file (lambda (title) title)
@@ -618,10 +667,15 @@ If item at point is not org-roam specific, default to Org behaviour."
              (select-window org-roam-last-window))
     (find-file file)))
 
+(defun org-roam--get-backlinks (file)
+  (org-roam-sql [:select [file-from, file-to, properties] :from file-links
+                 :where (= file-to $s1)]
+                file))
+
 (defun org-roam-update (file-path)
   "Show the backlinks for given org file for file at `FILE-PATH'."
+  (org-roam--ensure-cache-built)
   (let* ((source-org-roam-directory org-roam-directory))
-    (org-roam--ensure-cache-built)
     (let ((buffer-title (org-roam--get-title-or-slug file-path)))
       (with-current-buffer org-roam-buffer
         ;; When dir-locals.el is used to override org-roam-directory,
@@ -640,24 +694,27 @@ If item at point is not org-roam specific, default to Org behaviour."
             (setq org-return-follows-link t)
             (insert
              (propertize buffer-title 'font-lock-face 'org-document-title))
-            (if-let ((backlinks (gethash file-path (org-roam--backward-links-cache))))
+            (if-let* ((backlinks (org-roam--get-backlinks file-path))
+                      (grouped-backlinks (--group-by (nth 0 it) backlinks)))
                 (progn
                   (insert (format "\n\n* %d Backlinks\n"
-                                  (hash-table-count backlinks)))
+                                  (length backlinks)))
-                  (maphash (lambda (file-from contents)
+                  (dolist (group grouped-backlinks)
-                             (insert (format "** [[file:%s][%s]]\n"
+                    (let ((file-from (car group))
-                                             file-from
+                          (bls (cdr group)))
-                                             (org-roam--get-title-or-slug file-from)))
+                      (insert (format "** [[file:%s][%s]]\n"
-                             (dolist (properties contents)
+                                      file-from
-                               (let ((content (propertize
+                                      (org-roam--get-title-or-slug file-from)))
-                                               (s-trim (s-replace "\n" " "
+                      (dolist (backlink bls)
-                                                                  (plist-get properties :content)))
+                        (pcase-let ((`(,file-from ,file-to ,props) backlink))
-                                               'font-lock-face 'org-block
+                          (insert (propertize
-                                               'help-echo "mouse-1: visit backlinked note"
+                                   (s-trim (s-replace "\n" " "
-                                               'file-from file-from
+                                                      (plist-get props :content)))
-                                               'file-from-point (plist-get properties :point))))
+                                   'font-lock-face 'org-block
-                                 (insert (format "%s \n\n" content)))))
+                                   'help-echo "mouse-1: visit backlinked note"
-                           backlinks))
+                                   'file-from file-from
+                                   'file-from-point (plist-get props :point)))
+                          (insert "\n\n"))))))
               (insert "\n\n* No backlinks!")))
           (read-only-mode 1))))))
 
@@ -667,24 +724,24 @@ If item at point is not org-roam specific, default to Org behaviour."
   (org-roam--ensure-cache-built)
   (with-temp-buffer
 	  (insert "digraph {\n")
-	  (dolist (file (org-roam--find-all-files))
+    (let ((rows (org-roam-sql [:select [file titles] :from titles])))
-	    (let ((title (org-roam--get-title-or-slug file)))
+      (dolist (row rows)
-		    (let ((shortened-title (s-truncate org-roam-graph-max-title-length title)))
+        (let* ((file (car row))
-		      (insert
+               (title (or (caadr row)
+                          (org-roam--path-to-slug file)))
+               (shortened-title (s-truncate org-roam-graph-max-title-length title)))
+          (insert
 		       (format "  \"%s\" [label=\"%s\", shape=%s, URL=\"roam://%s\", tooltip=\"%s\"];\n"
-				           title
+                   file
 				           shortened-title
 				           org-roam-graph-node-shape
 				           file
-				           title
+				           title)))))
-				           )))))
+    (let ((link-rows (org-roam-sql [:select :distinct [file-to file-from] :from file-links])))
-	  (maphash
+      (dolist (row link-rows)
-	   (lambda (from-link to-links)
+        (insert (format "  \"%s\" -> \"%s\";\n"
-		   (dolist (to-link to-links)
+                        (car row)
-		     (insert (format "  \"%s\" -> \"%s\";\n"
+						            (cadr row)))))
-						             (org-roam--get-title-or-slug from-link)
-						             (org-roam--get-title-or-slug to-link)))))
-	   (org-roam--forward-links-cache))
 	  (insert "}")
 	  (buffer-string)))
 
@@ -715,7 +772,6 @@ This needs to be quick/infrequent, because this is run at
     (when (and (or redisplay
                    (not (eq org-roam--current-buffer buffer)))
                (eq 'visible (org-roam--current-visibility))
-               (org-roam--cache-initialized-p)
                (buffer-local-value 'buffer-file-truename buffer))
       (setq org-roam--current-buffer buffer)
       (org-roam-update (expand-file-name
@@ -740,13 +796,8 @@ Applies `org-roam-link-face' if PATH correponds to a Roam file."
     (setq org-roam-last-window (get-buffer-window))
     (add-hook 'post-command-hook #'org-roam--maybe-update-buffer nil t)
     (add-hook 'after-save-hook #'org-roam--update-cache nil t)
-    (if (org-roam--cache-initialized-p)
+    (org-roam--setup-file-links)
-        (org-roam--setup-found-file)
+    (org-roam--maybe-update-buffer :redisplay nil)))
-      (org-roam--build-cache-async
-       (let ((buf (buffer-name)))
-         #'(lambda ()
-             (with-current-buffer buf
-               (org-roam--setup-found-file))))))))
 
 (defun org-roam--setup-file-links ()
   "Set up `file:' Org links with org-roam-link-face."
@@ -759,11 +810,6 @@ This sets `file:' Org links to have the org-link face."
   (unless (version< org-version "9.2")
     (org-link-set-parameters "file" :face 'org-link)))
 
-(defun org-roam--setup-found-file ()
-  "Setup a buffer recognized via the \"find-file-hook\"."
-  (org-roam--setup-file-links)
-  (org-roam--maybe-update-buffer :redisplay nil))
-
 (defvar org-roam-mode-map
   (make-sparse-keymap)
   "Keymap for org-roam commands.")
@@ -778,9 +824,10 @@ This sets `file:' Org links to have the org-link face."
              (not (auto-save-file-name-p new-file))
              (org-roam--org-roam-file-p new-file))
     (org-roam--ensure-cache-built)
-    (org-roam--clear-file-from-cache file)
+    (let* ((files-to-rename (org-roam-sql [:select :distinct [file-from]
-
+                                           :from file-links
-    (let* ((files (gethash file (org-roam--backward-links-cache) nil))
+                                           :where (= file-to $s1)]
+                                          file))
            (path (file-truename file))
            (new-path (file-truename new-file))
            (slug (org-roam--get-title-or-slug file))
@@ -788,27 +835,31 @@ This sets `file:' Org links to have the org-link face."
            (new-slug (or (org-roam--get-title-from-cache path)
                          (org-roam--get-title-or-slug new-path)))
            (new-title (format org-roam-link-title-format new-slug)))
-      (when files
+      (org-roam--clear-file-from-cache file)
-        (maphash (lambda (file-from props)
+      (dolist (file-from files-to-rename)
-                   (let* ((file-dir (file-name-directory file-from))
+        (let* ((file-from (car file-from))
-                          (relative-path (file-relative-name new-path file-dir))
+               (file-from (if (string-equal (file-truename file-from)
-                          (old-relative-path (file-relative-name path file-dir))
+                                            path)
-                          (slug-regex (regexp-quote (format "[[file:%s][%s]]" old-relative-path old-title)))
+                              new-path
-                          (named-regex (concat
+                            file-from))
-                                        (regexp-quote (format "[[file:%s][" old-relative-path))
+               (file-dir (file-name-directory file-from))
-                                        "\\(.*\\)"
+               (relative-path (file-relative-name new-path file-dir))
-                                        (regexp-quote "]]"))))
+               (old-relative-path (file-relative-name path file-dir))
-                     (with-temp-file file-from
+               (slug-regex (regexp-quote (format "[[file:%s][%s]]" old-relative-path old-title)))
-                       (insert-file-contents file-from)
+               (named-regex (concat
-                       (while (re-search-forward slug-regex  nil t)
+                             (regexp-quote (format "[[file:%s][" old-relative-path))
-                         (replace-match (format "[[file:%s][%s]]" relative-path new-title)))
+                             "\\(.*\\)"
-                       (goto-char (point-min))
+                             (regexp-quote "]]"))))
-                       (while (re-search-forward named-regex nil t)
+          (with-temp-file file-from
-                         (replace-match (format "[[file:%s][\\1]]" relative-path))))
+            (insert-file-contents file-from)
-                     (save-window-excursion
+            (while (re-search-forward slug-regex nil t)
-                       (find-file file-from)
+              (replace-match (format "[[file:%s][%s]]" relative-path new-title)))
-                       (org-roam--update-cache))))
+            (goto-char (point-min))
-                 files))
+            (while (re-search-forward named-regex nil t)
+              (replace-match (format "[[file:%s][\\1]]" relative-path))))
+          (save-window-excursion
+            (find-file file-from)
+            (org-roam--update-cache))))
       (save-window-excursion
         (find-file new-path)
         (org-roam--update-cache)))))
@@ -818,7 +869,7 @@ This sets `file:' Org links to have the org-link face."
   "Minor mode for Org-roam.
 
 When called interactively, toggle `org-roam-mode'. with prefix ARG, enable `org-roam-mode'
-if ARG is posiwive, otherwise disable it.
+if ARG is positive, otherwise disable it.
 
 When called from Lisp, enable `org-roam-mode' if ARG is omitted, nil, or positive.
 If ARG is `toggle', toggle `org-roam-mode'. Otherwise, behave as if called interactively."
@@ -829,15 +880,16 @@ If ARG is `toggle', toggle `org-roam-mode'. Otherwise, behave as if called inter
   :global t
   (cond
    (org-roam-mode
-    (unless (org-roam--cache-initialized-p)
-      (org-roam--build-cache-async))
     (add-hook 'find-file-hook #'org-roam--find-file-hook-function)
+    (add-hook 'kill-emacs-hook #'org-roam--db-close-all)
     (advice-add 'rename-file :after #'org-roam--rename-file-advice)
     (advice-add 'delete-file :before #'org-roam--delete-file-advice))
    (t
     (remove-hook 'find-file-hook #'org-roam--find-file-hook-function)
+    (remove-hook 'kill-emacs-hook #'org-roam--db-close-all)
     (advice-remove 'rename-file #'org-roam--rename-file-advice)
     (advice-remove 'delete-file #'org-roam--delete-file-advice)
+    (org-roam--db-close-all)
     ;; Disable local hooks for all org-roam buffers
     (dolist (buf (org-roam--get-roam-buffers))
       (with-current-buffer buf
@@ -845,8 +897,6 @@ If ARG is `toggle', toggle `org-roam-mode'. Otherwise, behave as if called inter
         (remove-hook 'post-command-hook #'org-roam--maybe-update-buffer t)
         (remove-hook 'after-save-hook #'org-roam--update-cache t))))))
 
-(provide 'org-roam)
-
 ;;; Show/hide the org-roam buffer
 (define-inline org-roam--current-visibility ()
   "Return whether the current visibility state of the org-roam buffer.
@@ -889,6 +939,9 @@ Valid states are 'visible, 'exists and 'none."
     ('visible (delete-window (get-buffer-window org-roam-buffer)))
     ('exists (org-roam--setup-buffer))
     ('none (org-roam--setup-buffer))))
+
+;;; -
+(provide 'org-roam)
 ;;; org-roam.el ends here
 
 ;; Local Variables:

tests/roam-files-multi/mf1.org

@@ -1,7 +0,0 @@
-#+TITLE: Multi-File 1
-
-link to [[file:nested/mf1.org][Nested Multi-File 1]]
-link to [[file:mf2.org][Multi-File 2]]
-
-Arbitrary [[https://google.com][HTML]] link
-Arbitrary text

tests/roam-files-multi/mf2.org

@@ -1,3 +0,0 @@
-#+TITLE: Multi-File 2
-
-This file has no links.

tests/roam-files-multi/mf3.org

@@ -1,5 +0,0 @@
-#+TITLE: Multi-File 3
-
-This file has a link to an file with no title.
-
-[[file:multi-no-title.org][multi-no-title]]

tests/roam-files-multi/multi-no-title.org

@@ -1 +0,0 @@
-no title in this file :O

tests/roam-files-multi/nested/mf1.org

@@ -1,4 +0,0 @@
-#+TITLE: Nested Multi-File 1
-
-Link to [[file:mf2.org][Nested Multi-File 2]]
-Link to [[file:../mf1.org][Mulit-File 1]]

tests/roam-files-multi/nested/mf2.org

@@ -1,3 +0,0 @@
-#+TITLE: Nested Multi-File 2
-
-Link to [[file:mf1.org][Nested Multi-File 1]]

tests/roam-files/bar.org

@@ -0,0 +1,3 @@
+#+TITLE: Bar
+
+This is file bar. Bar links to [[file:nested/bar.org][Nested Bar]].

tests/roam-files/f1.org

@@ -1,7 +0,0 @@
-#+TITLE: File 1
-
-link to [[file:nested/f1.org][Nested File 1]] 
-link to [[file:f2.org][File 2]]
-
-Arbitrary [[https://google.com][HTML]] link
-Arbitrary text

tests/roam-files/f3.org

@@ -1,5 +0,0 @@
-#+TITLE: File 3
-
-This file has a link to an file with no title.
-
-[[file:no-title.org][no-title]]

tests/roam-files/foo.org

@@ -0,0 +1,8 @@
+#+TITLE: Foo
+
+This is the foo file. It contains a link to [[file:bar.org][Bar]].
+
+To make the tests more robust, here are some arbitrary links:
+
+- [[https:google.com][Google]]
+- [[mailto:foo@john.com][mail to foo]]

tests/roam-files/nested/bar.org

@@ -0,0 +1,3 @@
+#+TITLE: Nested Bar
+
+This file is nested, 1 level deeper. It links to both [[file:../foo.org][Foo]] and [[file:foo.org][Nested Foo]].

tests/roam-files/nested/f1.org

@@ -1,4 +0,0 @@
-#+TITLE: Nested File 1
-
-Link to [[file:f2.org][Nested File 2]]
-Link to [[file:../f1.org][File 1]]

tests/roam-files/nested/f2.org

@@ -1,3 +0,0 @@
-#+TITLE: Nested File 2
-
-Link to [[file:f1.org][Nested File 1]]

tests/roam-files/nested/foo.org

@@ -1,3 +1,3 @@
-#+TITLE: File 2
+#+TITLE: Nested Foo
 
 This file has no links.

tests/roam-files/no-title.org

@@ -1 +1,3 @@
 no title in this file :O
+
+links to itself, with no title: [[file:no-title.org][no-title]]

tests/roam-files/unlinked.org

@@ -0,0 +1,3 @@
+#+TITLE: Unlinked
+
+Nothing links here :(

tests/test-org-roam.el

@@ -29,6 +29,7 @@
 (require 'buttercup)
 (require 'with-simulated-input)
 (require 'org-roam)
+(require 'org-roam-db)
 (require 'dash)
 
 (defun abs-path (file-path)
@@ -42,320 +43,262 @@
 (defvar org-roam--tests-directory (file-truename (concat default-directory "tests/roam-files"))
   "Directory containing org-roam test org files.")
 
-(defvar org-roam--tests-multi (file-truename (concat default-directory "tests/roam-files-multi"))
-  "Directory containing org-roam test org files.")
-
 (defun org-roam--test-init ()
+  (org-roam--db-close)
   (let ((original-dir org-roam--tests-directory)
         (new-dir (expand-file-name (make-temp-name "org-roam") temporary-file-directory)))
     (copy-directory original-dir new-dir)
     (setq org-roam-directory new-dir)
-    (setq org-roam-mute-cache-build t))
+    (org-roam-mode +1)))
-  (org-roam-mode +1))
-
-(defun org-roam--test-multi-init ()
-  (let ((original-dir-1 org-roam--tests-directory)
-        (original-dir-2 org-roam--tests-multi)
-        (new-dir-1 (expand-file-name (make-temp-name "org-roam") temporary-file-directory))
-        (new-dir-2 (expand-file-name (make-temp-name "org-roam") temporary-file-directory)))
-    (copy-directory original-dir-1 new-dir-1)
-    (copy-directory original-dir-2 new-dir-2)
-    (setq org-roam-directory new-dir-1)
-    (setq org-roam-directory2 new-dir-2)
-    (setq org-roam-mute-cache-build t))
-  (org-roam-mode +1))
-
-(defun org-roam--test-build-cache ()
-  "Builds the caches synchronously."
-  (let ((cache (org-roam--build-cache org-roam-directory)))
-    (let ((obj (org-roam--get-directory-cache)))
-      (oset obj initialized t)
-      (oset obj forward-links (plist-get cache :forward))
-      (oset obj backward-links (plist-get cache :backward))
-      (oset obj titles (plist-get cache :titles))
-      (oset obj refs (plist-get cache :refs)))))
 
 ;;; Tests
-(describe "org-roam--build-cache-async"
+(describe "org-roam-build-cache"
           (it "initializes correctly"
-              (org-roam--clear-cache)
+              (org-roam--test-init)
-              (org-roam--test-multi-init)
+              (org-roam-build-cache)
-              (expect (org-roam--cache-initialized-p) :to-be nil)
-              (expect (hash-table-count (org-roam--forward-links-cache)) :to-be 0)
-              (expect (hash-table-count (org-roam--backward-links-cache)) :to-be 0)
-              (expect (hash-table-count (org-roam--titles-cache)) :to-be 0)
 
-              (org-roam--build-cache-async)
+              ;; Cache
-              (sleep-for 3) ;; Because it's async
+              (expect (caar (org-roam-sql [:select (funcall count) :from files])) :to-be 8)
+              (expect (caar (org-roam-sql [:select (funcall count) :from file-links])) :to-be 5)
+              (expect (caar (org-roam-sql [:select (funcall count) :from titles])) :to-be 8)
+              (expect (caar (org-roam-sql [:select (funcall count) :from titles
+                                                   :where titles :is-null])) :to-be 2)
+              (expect (caar (org-roam-sql [:select (funcall count) :from refs])) :to-be 1)
 
-              ;; Caches should be populated
+              ;; TODO Test files
-              (expect (org-roam--cache-initialized-p) :to-be t)
-              (expect (hash-table-count (org-roam--forward-links-cache)) :to-be 4)
-              (expect (hash-table-count (org-roam--backward-links-cache)) :to-be 5)
-              (expect (hash-table-count (org-roam--titles-cache)) :to-be 6)
-              (expect (hash-table-count (org-roam--refs-cache)) :to-be 1)
 
-              ;; Forward cache
+              ;; Links -- File-from
-              (let ((f1 (gethash (abs-path "f1.org")
+              (expect (caar (org-roam-sql [:select (funcall count) :from file-links
-                                 (org-roam--forward-links-cache)))
+                                                   :where (= file-from $s1)]
-                    (f2 (gethash (abs-path "f2.org")
+                                          (abs-path "foo.org"))) :to-be 1)
-                                 (org-roam--forward-links-cache)))
+              (expect (caar (org-roam-sql [:select (funcall count) :from file-links
-                    (nested-f1 (gethash (abs-path "nested/f1.org")
+                                                   :where (= file-from $s1)]
-                                        (org-roam--forward-links-cache)))
+                                          (abs-path "nested/bar.org"))) :to-be 2)
-                    (nested-f2 (gethash (abs-path "nested/f2.org")
-                                        (org-roam--forward-links-cache)))
-                    (expected-f1 (list (abs-path "nested/f1.org")
-                                       (abs-path "f2.org")))
-                    (expected-nested-f1 (list (abs-path "nested/f2.org")
-                                              (abs-path "f1.org")))
-                    (expected-nested-f2 (list (abs-path "nested/f1.org"))))
 
-                (expect f1 :to-have-same-items-as expected-f1)
+              ;; Links -- File-to
-                (expect f2 :to-be nil)
+              (expect (caar (org-roam-sql [:select (funcall count) :from file-links
-                (expect nested-f1 :to-have-same-items-as expected-nested-f1)
+                                                   :where (= file-to $s1)]
-                (expect nested-f2 :to-have-same-items-as expected-nested-f2))
+                                          (abs-path "nested/foo.org"))) :to-be 1)
+              (expect (caar (org-roam-sql [:select (funcall count) :from file-links
+                                                   :where (= file-to $s1)]
+                                          (abs-path "nested/bar.org"))) :to-be 1)
+              (expect (caar (org-roam-sql [:select (funcall count) :from file-links
+                                                   :where (= file-to $s1)]
+                                          (abs-path "unlinked.org"))) :to-be 0)
+              ;; TODO Test titles
+              (expect (org-roam-sql [:select * :from titles])
+                      :to-have-same-items-as
+                      (list (list (abs-path "alias.org")
+                                  (list "t1" "a1" "a 2"))
+                            (list (abs-path "bar.org")
+                                  (list "Bar"))
+                            (list (abs-path "foo.org")
+                                  (list "Foo"))
+                            (list (abs-path "nested/bar.org")
+                                  (list "Nested Bar"))
+                            (list (abs-path "nested/foo.org")
+                                  (list "Nested Foo"))
+                            (list (abs-path "no-title.org") nil)
+                            (list (abs-path "web_ref.org") nil)
+                            (list (abs-path "unlinked.org")
+                                  (list "Unlinked"))))
 
-              ;; Backward cache
+              (expect (org-roam-sql [:select * :from refs])
-              (let ((f1 (hash-table-keys (gethash (abs-path "f1.org")
+                      :to-have-same-items-as
-                                                  (org-roam--backward-links-cache))))
+                      (list (list "https://google.com/" (abs-path "web_ref.org"))))
-                    (f2 (hash-table-keys (gethash (abs-path "f2.org")
-                                                  (org-roam--backward-links-cache))))
-                    (nested-f1 (hash-table-keys
-                                (gethash (abs-path "nested/f1.org")
-                                         (org-roam--backward-links-cache))))
-                    (nested-f2 (hash-table-keys
-                                (gethash (abs-path "nested/f2.org")
-                                         (org-roam--backward-links-cache))))
-                    (expected-f1 (list (abs-path "nested/f1.org")))
-                    (expected-f2 (list (abs-path "f1.org")))
-                    (expected-nested-f1 (list (abs-path "nested/f2.org")
-                                              (abs-path "f1.org")))
-                    (expected-nested-f2 (list (abs-path "nested/f1.org"))))
-                (expect f1 :to-have-same-items-as expected-f1)
-                (expect f2 :to-have-same-items-as expected-f2)
-                (expect nested-f1 :to-have-same-items-as expected-nested-f1)
-                (expect nested-f2 :to-have-same-items-as expected-nested-f2))
 
-              ;; Titles Cache
+              ;; Expect rebuilds to be really quick (nothing changed)
-              (expect (gethash (abs-path "f1.org")
+              (expect (org-roam-build-cache)
-                               (org-roam--titles-cache)) :to-equal (list "File 1"))
+                      :to-equal
-              (expect (gethash (abs-path "f2.org")
+                      (list :files 0 :links 0 :titles 0 :refs 0 :deleted 0))))
-                               (org-roam--titles-cache)) :to-equal (list "File 2"))
-              (expect (gethash (abs-path "nested/f1.org")
-                               (org-roam--titles-cache)) :to-equal (list "Nested File 1"))
-              (expect (gethash (abs-path "nested/f2.org")
-                               (org-roam--titles-cache)) :to-equal (list "Nested File 2"))
-              (expect (gethash (abs-path "alias.org")
-                               (org-roam--titles-cache)) :to-equal (list "t1" "a1" "a 2"))
-              (expect (gethash (abs-path "no-title.org")
-                               (org-roam--titles-cache)) :to-be nil)
-
-              ;; Refs Cache
-              (expect (gethash "https://google.com/"
-                               (org-roam--refs-cache)) :to-equal (abs-path "web_ref.org"))
-
-              ;; Multi
-              (let ((org-roam-directory org-roam-directory2))
-                (org-roam--build-cache-async)
-                (sleep-for 3) ;; Because it's async
-
-                ;; Caches should be populated
-                (expect (org-roam--cache-initialized-p) :to-be t)
-                (expect (hash-table-count (org-roam--forward-links-cache)) :to-be 4)
-                (expect (hash-table-count (org-roam--backward-links-cache)) :to-be 5)
-                (expect (hash-table-count (org-roam--titles-cache)) :to-be 5)
-
-                ;; Forward cache
-                (let ((mf1 (gethash (abs-path "mf1.org")
-                                    (org-roam--forward-links-cache)))
-                      (mf2 (gethash (abs-path "mf2.org")
-                                    (org-roam--forward-links-cache)))
-                      (nested-mf1 (gethash (abs-path "nested/mf1.org")
-                                           (org-roam--forward-links-cache)))
-                      (nested-mf2 (gethash (abs-path "nested/mf2.org")
-                                           (org-roam--forward-links-cache)))
-                      (expected-mf1 (list (abs-path "nested/mf1.org")
-                                          (abs-path "mf2.org")))
-                      (expected-nested-mf1 (list (abs-path "nested/mf2.org")
-                                                 (abs-path "mf1.org")))
-                      (expected-nested-mf2 (list (abs-path "nested/mf1.org"))))
-
-                  (expect mf1 :to-have-same-items-as expected-mf1)
-                  (expect mf2 :to-be nil)
-                  (expect nested-mf1 :to-have-same-items-as expected-nested-mf1)
-                  (expect nested-mf2 :to-have-same-items-as expected-nested-mf2))
-
-                ;; Backward cache
-                (let ((mf1 (hash-table-keys
-                            (gethash (abs-path "mf1.org")
-                                     (org-roam--backward-links-cache))))
-                      (mf2 (hash-table-keys
-                            (gethash (abs-path "mf2.org")
-                                     (org-roam--backward-links-cache))))
-                      (nested-mf1 (hash-table-keys
-                                   (gethash (abs-path "nested/mf1.org")
-                                            (org-roam--backward-links-cache))))
-                      (nested-mf2 (hash-table-keys
-                                   (gethash (abs-path "nested/mf2.org")
-                                            (org-roam--backward-links-cache))))
-                      (expected-mf1 (list (abs-path "nested/mf1.org")))
-                      (expected-mf2 (list (abs-path "mf1.org")))
-                      (expected-nested-mf1 (list (abs-path "nested/mf2.org")
-                                                 (abs-path "mf1.org")))
-                      (expected-nested-mf2 (list (abs-path "nested/mf1.org"))))
-                  (expect mf1 :to-have-same-items-as expected-mf1)
-                  (expect mf2 :to-have-same-items-as expected-mf2)
-                  (expect nested-mf1 :to-have-same-items-as expected-nested-mf1)
-                  (expect nested-mf2 :to-have-same-items-as expected-nested-mf2))
-
-                ;; Titles Cache
-                (expect (gethash (abs-path "mf1.org")
-                                 (org-roam--titles-cache))
-                        :to-equal (list "Multi-File 1"))
-                (expect (gethash (abs-path "mf2.org")
-                                 (org-roam--titles-cache))
-                        :to-equal (list "Multi-File 2"))
-                (expect (gethash (abs-path "nested/mf1.org")
-                                 (org-roam--titles-cache))
-                        :to-equal (list "Nested Multi-File 1"))
-                (expect (gethash (abs-path "nested/mf2.org")
-                                 (org-roam--titles-cache))
-                        :to-equal (list "Nested Multi-File 2"))
-                (expect (gethash (abs-path "no-title.org")
-                                 (org-roam--titles-cache))
-                        :to-be nil))))
 
 (describe "org-roam-insert"
           (before-each
            (org-roam--test-init)
            (org-roam--clear-cache)
-           (org-roam--test-build-cache))
+           (org-roam-build-cache))
 
-          (it "temp1 -> f1"
+          (it "temp1 -> foo"
               (let ((buf (org-roam--test-find-new-file "temp1.org")))
                 (with-current-buffer buf
                   (with-simulated-input
-                   "File SPC 1 RET"
+                   "Foo RET"
                    (org-roam-insert nil))))
-              (expect (buffer-string) :to-match (regexp-quote "file:f1.org")))
+              (expect (buffer-string) :to-match (regexp-quote "file:foo.org")))
 
-          (it "temp2 -> nested/f1"
+          (it "temp2 -> nested/foo"
               (let ((buf (org-roam--test-find-new-file "temp2.org")))
                 (with-current-buffer buf
                   (with-simulated-input
-                   "Nested SPC File SPC 1 RET"
+                   "Nested SPC Foo RET"
                    (org-roam-insert nil))))
-              (expect (buffer-string) :to-match (regexp-quote "file:nested/f1.org")))
+              (expect (buffer-string) :to-match (regexp-quote "file:nested/foo.org")))
 
-          (it "nested/temp3 -> f1"
+          (it "nested/temp3 -> foo"
               (let ((buf (org-roam--test-find-new-file "nested/temp3.org")))
                 (with-current-buffer buf
                   (with-simulated-input
-                   "File SPC 1 RET"
+                   "Foo RET"
                    (org-roam-insert nil))))
-              (expect (buffer-string) :to-match (regexp-quote "file:../f1.org")))
+              (expect (buffer-string) :to-match (regexp-quote "file:../foo.org")))
 
-          (it "a/b/temp4 -> nested/f1"
+          (it "a/b/temp4 -> nested/foo"
               (let ((buf (org-roam--test-find-new-file "a/b/temp4.org")))
                 (with-current-buffer buf
                   (with-simulated-input
-                   "Nested SPC File SPC 1 RET"
+                   "Nested SPC Foo RET"
                    (org-roam-insert nil))))
-              (expect (buffer-string) :to-match (regexp-quote "file:../../nested/f1.org"))))
+              (expect (buffer-string) :to-match (regexp-quote "file:../../nested/foo.org"))))
 
 (describe "rename file updates cache"
           (before-each
            (org-roam--test-init)
            (org-roam--clear-cache)
-           (org-roam--test-build-cache))
+           (org-roam-build-cache))
 
-          (it "f1 -> new_f1"
+          (it "foo -> new_foo"
-              (rename-file (abs-path "f1.org")
+              (rename-file (abs-path "foo.org")
-                           (abs-path "new_f1.org"))
+                           (abs-path "new_foo.org"))
               ;; Cache should be cleared of old file
-              (expect (gethash (abs-path "f1.org") (org-roam--forward-links-cache)) :to-be nil)
+              (expect (caar (org-roam-sql [:select (funcall count)
-              (expect (->> (org-roam--backward-links-cache)
+                                                   :from titles
-                           (gethash (abs-path "nested/f1.org"))
+                                                   :where (= file $s1)]
-                           (hash-table-keys)
+                                          (abs-path "foo.org"))) :to-be 0)
-                           (member (abs-path "f1.org"))) :to-be nil)
+              (expect (caar (org-roam-sql [:select (funcall count)
+                                                   :from refs
+                                                   :where (= file $s1)]
+                                          (abs-path "foo.org"))) :to-be 0)
+              (expect (caar (org-roam-sql [:select (funcall count)
+                                                   :from file-links
+                                                   :where (= file-from $s1)]
+                                          (abs-path "foo.org"))) :to-be 0)
 
-              (expect (->> (org-roam--forward-links-cache)
+              ;; Cache should be updated
-                           (gethash (abs-path "new_f1.org"))) :not :to-be nil)
+              (expect (org-roam-sql [:select [file-to]
-
+                                             :from file-links
-              (expect (->> (org-roam--forward-links-cache)
+                                             :where (= file-from $s1)]
-                           (gethash (abs-path "new_f1.org"))
+                                    (abs-path "new_foo.org"))
-                           (member (abs-path "nested/f1.org"))) :not :to-be nil)
+                      :to-have-same-items-as
+                      (list (list (abs-path "bar.org"))))
+              (expect (org-roam-sql [:select [file-from]
+                                             :from file-links
+                                             :where (= file-to $s1)]
+                                    (abs-path "new_foo.org"))
+                      :to-have-same-items-as
+                      (list (list (abs-path "nested/bar.org"))))
 
               ;; Links are updated
               (expect (with-temp-buffer
-                        (insert-file-contents (abs-path "nested/f1.org"))
+                        (insert-file-contents (abs-path "nested/bar.org"))
-                        (buffer-string)) :to-match (regexp-quote "[[file:../new_f1.org][File 1]]")))
+                        (buffer-string))
+                      :to-match
+                      (regexp-quote "[[file:../new_foo.org][Foo]]")))
 
-          (it "f1 -> f1 with spaces"
+          (it "foo -> foo with spaces"
-              (rename-file (abs-path "f1.org")
+              (rename-file (abs-path "foo.org")
-                           (abs-path "f1 with spaces.org"))
+                           (abs-path "foo with spaces.org"))
               ;; Cache should be cleared of old file
-              (expect (gethash (abs-path "f1.org")  (org-roam--forward-links-cache)) :to-be nil)
+              (expect (caar (org-roam-sql [:select (funcall count)
-              (expect (->> (org-roam--backward-links-cache)
+                                                   :from titles
-                           (gethash (abs-path "nested/f1.org"))
+                                                   :where (= file $s1)]
-                           (hash-table-keys)
+                                          (abs-path "foo.org"))) :to-be 0)
-                           (member (abs-path "f1.org"))) :to-be nil)
+              (expect (caar (org-roam-sql [:select (funcall count)
+                                                   :from refs
+                                                   :where (= file $s1)]
+                                          (abs-path "foo.org"))) :to-be 0)
+              (expect (caar (org-roam-sql [:select (funcall count)
+                                                   :from file-links
+                                                   :where (= file-from $s1)]
+                                          (abs-path "foo.org"))) :to-be 0)
+
+              ;; Cache should be updated
+              (expect (org-roam-sql [:select [file-to]
+                                             :from file-links
+                                             :where (= file-from $s1)]
+                                    (abs-path "foo with spaces.org"))
+                      :to-have-same-items-as
+                      (list (list (abs-path "bar.org"))))
+              (expect (org-roam-sql [:select [file-from]
+                                             :from file-links
+                                             :where (= file-to $s1)]
+                                    (abs-path "foo with spaces.org"))
+                      :to-have-same-items-as
+                      (list (list (abs-path "nested/bar.org"))))
+
               ;; Links are updated
               (expect (with-temp-buffer
-                        (insert-file-contents (abs-path "nested/f1.org"))
+                        (insert-file-contents (abs-path "nested/bar.org"))
-                        (buffer-string)) :to-match (regexp-quote "[[file:../f1 with spaces.org][File 1]]")))
+                        (buffer-string))
+                      :to-match
+                      (regexp-quote "[[file:../foo with spaces.org][Foo]]")))
 
           (it "no-title -> meaningful-title"
               (rename-file (abs-path "no-title.org")
                            (abs-path "meaningful-title.org"))
               ;; File has no forward links
-              (expect (gethash (abs-path "no-title.org")  (org-roam--forward-links-cache)) :to-be nil)
+              (expect (caar (org-roam-sql [:select (funcall count)
-              (expect (gethash (abs-path "meaningful-title.org")
+                                                   :from file-links
-                               (org-roam--forward-links-cache)) :to-be nil)
+                                                   :where (= file-from $s1)]
-
+                                          (abs-path "no-title.org"))) :to-be 0)
-              (expect (->> (org-roam--forward-links-cache)
+              (expect (caar (org-roam-sql [:select (funcall count)
-                           (gethash (abs-path "f3.org"))
+                                                   :from file-links
-                           (member (abs-path "no-title.org"))) :to-be nil)
+                                                   :where (= file-from $s1)]
-
+                                          (abs-path "meaningful-title.org"))) :to-be 1)
-              (expect (->> (org-roam--forward-links-cache)
-                           (gethash (abs-path "f3.org"))
-                           (member (abs-path "meaningful-title.org"))) :not :to-be nil)
 
               ;; Links are updated with the appropriate name
               (expect (with-temp-buffer
-                        (insert-file-contents (abs-path "f3.org"))
+                        (insert-file-contents (abs-path "meaningful-title.org"))
-                        (buffer-string)) :to-match (regexp-quote "[[file:meaningful-title.org][meaningful-title]]")))
+                        (buffer-string))
+                      :to-match
+                      (regexp-quote "[[file:meaningful-title.org][meaningful-title]]")))
 
           (it "web_ref -> hello"
-              (expect (->> (org-roam--refs-cache)
+              (expect (org-roam-sql
-                           (gethash "https://google.com/"))
+                       [:select [file] :from refs
-                      :to-equal (abs-path "web_ref.org"))
+                                :where (= ref $s1)]
+                       "https://google.com/")
+                      :to-equal
+                      (list (list (abs-path "web_ref.org"))))
               (rename-file (abs-path "web_ref.org")
                            (abs-path "hello.org"))
-              (expect (->> (org-roam--refs-cache)
+              (expect (org-roam-sql
-                           (gethash "https://google.com/"))
+                       [:select [file] :from refs
-                      :to-equal (abs-path "hello.org"))))
+                                :where (= ref $s1)]
+                       "https://google.com/")
+                      :to-equal (list (list (abs-path "hello.org"))))
+              (expect (caar (org-roam-sql
+                             [:select [ref] :from refs
+                                      :where (= file $s1)]
+                             (abs-path "web_ref.org")))
+                      :to-equal nil)))
 
 (describe "delete file updates cache"
           (before-each
            (org-roam--test-init)
            (org-roam--clear-cache)
-           (org-roam--test-build-cache))
+           (org-roam-build-cache)
-          (it "delete f1"
+           (sleep-for 1))
-              (delete-file (abs-path "f1.org"))
+
-              (expect (->> (org-roam--forward-links-cache)
+          (it "delete foo"
-                           (gethash (abs-path "f1.org"))) :to-be nil)
+              (delete-file (abs-path "foo.org"))
-              (expect (->> (org-roam--backward-links-cache)
+              (expect (caar (org-roam-sql [:select (funcall count)
-                           (gethash (abs-path "nested/f1.org"))
+                                                   :from titles
-                           (gethash (abs-path "f1.org"))) :to-be nil)
+                                                   :where (= file $s1)]
-              (expect (->> (org-roam--backward-links-cache)
+                                          (abs-path "foo.org"))) :to-be 0)
-                           (gethash (abs-path "nested/f1.org"))
+              (expect (caar (org-roam-sql [:select (funcall count)
-                           (gethash (abs-path "nested/f2.org"))) :not :to-be nil))
+                                                   :from refs
+                                                   :where (= file $s1)]
+                                          (abs-path "foo.org"))) :to-be 0)
+              (expect (caar (org-roam-sql [:select (funcall count)
+                                                   :from file-links
+                                                   :where (= file-from $s1)]
+                                          (abs-path "foo.org"))) :to-be 0))
+
           (it "delete web_ref"
-              (expect (->> (org-roam--refs-cache)
+              (expect (org-roam-sql [:select * :from refs])
-                           (gethash "https://google.com/"))
+                      :to-have-same-items-as
-                      :to-equal (abs-path "web_ref.org"))
+                      (list (list "https://google.com/" (abs-path "web_ref.org"))))
               (delete-file (abs-path "web_ref.org"))
-              (expect (->> (org-roam--refs-cache)
+              (expect (org-roam-sql [:select * :from refs])
-                           (gethash "https://google.com/"))
+                      :to-have-same-items-as
-                      :to-be nil)))
+                      (list))))