v2.1.0 (#1792)

This is now the default behavior for `org-roam-find-noselect` and
`org-roam-node-visit`, unless FORCE parameter set to non-nil.

Co-authored-by: Duncan Burke <duncankburke@gmail.com>

.dir-locals.el

@@ -9,4 +9,5 @@
                               (org-with-point-at . 1)
                               (magit-insert-section . defun)
                               (magit-section-case . 0)
+                              (->> . 1)
                               (org-roam-with-file . 2)))))

CHANGELOG.md

@@ -1,9 +1,36 @@
 # Changelog
 
-## 1.2.4 (TBD)
+## 2.1.0
-
 ### Added
+- [#1709](https://github.com/org-roam/org-roam/pull/1709) added ability to specify default value in org-roam capture templates
+- [#1710](https://github.com/org-roam/org-roam/pull/1710) added `org-roam-extract-subtree`
+- [#1720](https://github.com/org-roam/org-roam/pull/1720) added `org-roam-db-update-on-save`
+- [#1758](https://github.com/org-roam/org-roam/pull/1758) added `org-roam-db-autosync-mode`, replacing `org-roam-setup` and `org-roam-teardown`
 
+### Removed
+- [#1716](https://github.com/org-roam/org-roam/pull/1716) helper function `org-roam-get-keyword` is now obsolete: prefer `org-collect-keywords`
+
+### Changed
+- [#1595](https://github.com/org-roam/org-roam/pull/1595), [#1724](https://github.com/org-roam/org-roam/pull/1724) Major refactoring and restructuring of the codebase
+- [#1655](https://github.com/org-roam/org-roam/pull/1655) improved org-roam contents preview
+- [#1741](https://github.com/org-roam/org-roam/pull/1741) expose `org-roam-capture-` keys in interactive commands
+- [#1786](https://github.com/org-roam/org-roam/pull/1786) org-roam-version now outputs commit hash if found
+- [#1788](https://github.com/org-roam/org-roam/pull/1788) the point is not moved if the node is already visited
+
+### Fixed
+- [#1608](https://github.com/org-roam/org-roam/pull/1608) migration: empty ROAM_REFS are now removed 
+- [#1609](https://github.com/org-roam/org-roam/pull/1609) migration: fixed file-link replacement
+- [#1653](https://github.com/org-roam/org-roam/pull/1653) migration: fixed tags migration
+- [#1694](https://github.com/org-roam/org-roam/pull/1694) core: nodes with no title are now skipped
+- [#1637](https://github.com/org-roam/org-roam/pull/1637) core: fix org-ref multi-cite links not being split
+- [#1651](https://github.com/org-roam/org-roam/pull/1651) core: fix org-roam-file-p crashing when there is no corresponding file
+- [#1705](https://github.com/org-roam/org-roam/pull/1705) core: fix for add/remove of file-level tags
+- [#1769](https://github.com/org-roam/org-roam/pull/1769) core: gracefully handle absence of `org-id-locations-file`
+- [#1713](https://github.com/org-roam/org-roam/pull/1713) capture: check for file-existence before template insertion
+- [#1725](https://github.com/org-roam/org-roam/pull/1725) db: always compute hash of encrypted file to prevent re-processing of encrypted files
+
+## 2.0.0
+### Added
 - [#1396](https://github.com/org-roam/org-roam/pull/1396) add option to choose between prepending, appending, and omitting `roam_tags` in file completion
 - [#1270](https://github.com/org-roam/org-roam/pull/1270) capture: create OLP if it does not exist. Removes need for OLP setup in `:head`.
 - [#1353](https://github.com/org-roam/org-roam/pull/1353) support file-level property drawers
@@ -11,6 +38,8 @@
 ### Changed
 
 - [#1352](https://github.com/org-roam/org-roam/pull/1352) prefer lower-case for roam_tag and roam_alias in interactive commands
+- [#1513](https://github.com/org-roam/org-roam/pull/1513) replaced hardcoded "svg" with defcustom org-roam-graph-filetype 
+- [#1540](https://github.com/org-roam/org-roam/pull/1540) allow `roam_tag` and `roam_alias` to be specified on multiple lines
 
 ### Fixed
 
@@ -25,6 +54,7 @@
 - [#1375](https://github.com/org-roam/org-roam/pull/1375) fix org-roam-protocol to use existing ref file
 - [#1403](https://github.com/org-roam/org-roam/issues/1403) fixed inconsistency between how we write and read props like alias and tags
 - [#1409](https://github.com/org-roam/org-roam/issues/1398) prevent inclusion of non-org-roam files in `org-roam-dailies--list-files`
+- [#1542](https://github.com/org-roam/org-roam/issues/1542) fix files not excluded when `org-roam-list-files-commands` is nil
 
 ## 1.2.3 (13-11-2020)
 

README.md

@@ -33,32 +33,178 @@ solution for anyone already using Org-mode for their personal wiki.
 
 ## Installation
 
-You can install `org-roam` using `package.el`:
+Down below you will find basic installation instructions for how to quickly
+install `org-roam` using various environments for various purposes. For more
+detailed information, please read the [manual][docs].
+
+### Using `package.el`
+<details>
+<summary>Toggle instuctions</summary>
+
+You can install `org-roam` from [MELPA](https://melpa.org/) or [MELPA
+Stable](https://stable.melpa.org/) using `package.el`:
 
 ```
 M-x package-install RET org-roam RET
 ```
 
-Here's a sample configuration with `use-package`:
+Here's a very basic sample for configuration of `org-roam` using `use-package`:
 
 ```emacs-lisp
 (use-package org-roam
-      :ensure t
+  :ensure t
-      :hook
+  :custom
-      (after-init . org-roam-mode)
+  (org-roam-directory (file-truename "/path/to/org-files/"))
-      :custom
+  :bind (("C-c n l" . org-roam-buffer-toggle)
-      (org-roam-directory "/path/to/org-files/")
+         ("C-c n f" . org-roam-node-find)
-      :bind (:map org-roam-mode-map
+         ("C-c n g" . org-roam-graph)
-              (("C-c n l" . org-roam)
+         ("C-c n i" . org-roam-node-insert)
-               ("C-c n f" . org-roam-find-file)
+         ("C-c n c" . org-roam-capture)
-               ("C-c n g" . org-roam-graph))
+         ;; Dailies
-              :map org-mode-map
+         ("C-c n j" . org-roam-dailies-capture-today))
-              (("C-c n i" . org-roam-insert))))
+  :config
+  (org-roam-db-autosync-mode)
+  ;; If using org-roam-protocol
+  (require 'org-roam-protocol))
 ```
 
-Org-roam requires sqlite to function. Org-roam optionally uses Graphviz for
+Note that the `file-truename` function is only necessary when you use symbolic
-graph-related functionality. It is recommended to install PCRE-enabled ripgrep
+link to `org-roam-directory`. Org-roam won't automatically resolve symbolic link
-for better performance and extended functionality.
+to the directory.
+</details>
+
+### Using `straight.el`
+<details>
+<summary>Toggle instuctions</summary>
+
+Installation from MELPA or MELPA Stable using `straight.el`:
+
+```emacs-lisp
+(straight-use-package 'org-roam)
+```
+
+Or with `use-package`:
+
+```emacs-lisp
+(use-package org-roam
+  :straight t
+  ...)
+```
+
+If you need to install the package directly from the source repository, instead
+of from MELPA, the next sample shows how to do so:
+
+```emacs-lisp
+(use-package org-roam
+  :straight (:host github :repo "org-roam/org-roam"
+             :files (:defaults "extensions/*"))
+  ...)
+```
+
+If you plan to use your own local fork for the development and contribution, the
+next sample will get you there:
+
+```emacs-lisp
+(use-package org-roam
+  :straight (:local-repo "/path/to/org-roam-fork"
+             :files (:defaults "extensions/*")
+             :build (:not compile))
+  ...)
+```
+</details>
+
+### Using Doom Emacs
+<details>
+<summary>Toggle instuctions</summary>
+
+Doom's `:lang org` module comes with support for `org-roam`, but it's not
+enabled by default. To activate it pass `+roam2` flag to `org` module in your
+`$DOOMDIR/init.el` (e.g. `(org +roam2)`), save the file and run `doom sync -u`
+in your shell.
+
+To provide better stability, Doom pins the package to a specific commit. If you
+need to unpin it *(not recommended doing that, request Doom to bump the package
+instead)* use the next in your `packages.el`:
+
+```emacs-lisp
+(unpin! org-roam)
+```
+
+If for some reasons you want to use a different recipe for `org-roam`, you can
+use the next form in your `packages.el` to install the package from a recipe
+repository (e.g. MELPA):
+
+```emacs-lisp
+(package! org-roam)
+```
+
+You can pass `:pin "commit hash"` to pin the package to a specific commit.
+
+With the next sample you can install the package directly from the source
+repository:
+
+```emacs-lisp
+(package! org-roam
+  :recipe (:host github :repo "org-roam/org-roam"
+           :files (:defaults "extensions/*")))
+```
+
+And if you plan to use your own local fork for the development or contribution,
+the next sample will get you there:
+
+```emacs-lisp
+(package! org-roam
+  :recipe (:local-repo "/path/to/org-roam-fork"
+           :files (:defaults "extensions/*")
+           :build (:not compile)))
+```
+</details>
+
+### Without a package manager
+<details>
+<summary>Toggle instructions</summary>
+
+To install the package without using a package manager you have the next two
+options:
+
+1. Install the package by cloning it with `git` from the source repository.
+2. Or install the package by downloading the latest [release
+   version](https://github.com/org-roam/org-roam/releases).
+   
+In both of the cases you will need to ensure that you have all the required
+dependencies. These include:
+
+- dash
+- f
+- s
+- org (9.4 is the minimal required version!)
+- emacsql
+- emacsql-sqlite
+- magit-section
+- filenotify-recursive
+
+After installing the package, you will need to properly setup `load-path` to the
+package:
+
+``` emacs-lisp
+(add-to-list 'load-path "/path/to/org-roam/")
+(add-to-list 'load-path "/path/to-org-roam/extensions/")
+```
+
+After which you should be able to resolve `(require 'org-roam)` call without any
+problems.
+
+Org-roam also comes with `.texi` files to integrate with Emacs' built-in Info
+system. Read the manual to find more details for how to install them manually.
+</details>
+
+## Getting Started
+
+[David Wilson](https://github.com/daviwil) of [System
+Crafters](https://www.youtube.com/c/SystemCrafters) has produced an introductory
+video that covers the basic commands:
+
+[![Getting Started with Org Roam - Build a Second Brain in Emacs](https://img.youtube.com/vi/AyhPmypHDEw/0.jpg)](https://www.youtube.com/watch?v=AyhPmypHDEw)
 
 ## Getting Help
 
@@ -80,6 +226,7 @@ it has not already been addressed on [GitHub][issues] or on
 - [Jethro Kuan](https://braindump.jethro.dev/)
   ([Source](https://github.com/jethrokuan/braindump/tree/master/org))
 - [Alexey Shmalko](https://braindump.rasen.dev/)
+- [Sidharth Arya](https://sidhartharya.github.io/braindump/index.html)
 
 ## Contributing
 

default.mk

@@ -44,16 +44,17 @@ HTMLDIRS  = $(PACKAGES)
 PDFFILES  = $(addsuffix .pdf,$(PACKAGES))
 EPUBFILES = $(addsuffix .epub,$(PACKAGES))
 
-ELS = org-roam-buffer.el
+ELS = org-roam.el
 ELS += org-roam-capture.el
 ELS += org-roam-compat.el
-ELS += org-roam-completion.el
-ELS += org-roam-dailies.el
 ELS += org-roam-db.el
-ELS += org-roam.el
+ELS += org-roam-mode.el
-ELS += org-roam-graph.el
+ELS += org-roam-node.el
-ELS += org-roam-macs.el
+ELS += org-roam-utils.el
-ELS += org-roam-protocol.el
+ELS += extensions/org-roam-dailies.el
+ELS += extensions/org-roam-graph.el
+ELS += extensions/org-roam-overlay.el
+ELS += extensions/org-roam-protocol.el
 ELCS = $(ELS:.el=.elc)
 ELMS = org-roam.el $(filter-out $(addsuffix .el,$(PACKAGES)),$(ELS))
 ELGS = org-roam-autoloads.el org-roam-version.el

doc/index.html

@@ -116,8 +116,8 @@
         <div class="row">
           <a
             class="content footer-links"
-            href="https://github.com/org-roam/org-roam-server"
+            href="https://github.com/org-roam/org-roam-ui"
-            >org-roam-server</a
+            >org-roam-ui</a
           >
         </div>
       </div>

extensions/org-roam-dailies.el

@@ -1,14 +1,14 @@
 ;;; org-roam-dailies.el --- Daily-notes for Org-roam -*- coding: utf-8; lexical-binding: t; -*-
 ;;;
-;; Copyright © 2020 Jethro Kuan <jethrokuan95@gmail.com>
+;; Copyright © 2020-2021 Jethro Kuan <jethrokuan95@gmail.com>
 ;; Copyright © 2020 Leo Vivier <leo.vivier+dev@gmail.com>
 
 ;; Author: Jethro Kuan <jethrokuan95@gmail.com>
 ;;      Leo Vivier <leo.vivier+dev@gmail.com>
 ;; URL: https://github.com/org-roam/org-roam
 ;; Keywords: org-mode, roam, convenience
-;; Version: 2.0.0
+;; Version: 2.1.0
-;; Package-Requires: ((emacs "26.1") (dash "2.13") (f "0.17.2") (org "9.4") (emacsql "3.0.0") (emacsql-sqlite3 "1.0.2") (magit-section "2.90.1"))
+;; Package-Requires: ((emacs "26.1") (dash "2.13") (f "0.17.2") (org-roam "2.1"))
 
 ;; This file is NOT part of GNU Emacs.
 
@@ -29,29 +29,28 @@
 
 ;;; Commentary:
 ;;
-;; This library provides functionality for creating daily-notes. This is a
+;; This extension provides functionality for creating daily-notes, or shortly
-;; concept borrowed from Roam Research.
+;; "dailies". Dailies implemented here as a unique node per unique file, where
+;; each file named after certain date and stored in `org-roam-dailies-directory'.
+;;
+;; One can use dailies for various purposes, e.g. journaling, fleeting notes,
+;; scratch notes and whatever else you can came up with.
 ;;
 ;;; Code:
-;;; Library Requires
-(require 'org-capture)
-(require 'org-roam-capture)
 (require 'f)
+(require 'dash)
+(require 'org-roam)
 
-;;;; Declarations
+;;; Faces
-(defvar org-roam-directory)
-(defvar org-roam-file-extensions)
-(declare-function org-roam--org-file-p        "org-roam")
-
-;;;; Faces
 (defface org-roam-dailies-calendar-note
   '((t :inherit (org-link) :underline nil))
   "Face for dates with a daily-note in the calendar."
   :group 'org-roam-faces)
 
-;;;; Customizable variables
+;;; Options
 (defcustom org-roam-dailies-directory "daily/"
-  "Path to daily-notes."
+  "Path to daily-notes.
+This path is relative to `org-roam-directory'."
   :group 'org-roam
   :type 'string)
 
@@ -61,11 +60,13 @@
   :type 'hook)
 
 (defcustom org-roam-dailies-capture-templates
-  '(("d" "default" entry
+  `(("d" "default" entry
      "* %?"
-     :if-new `(file+head ,(concat org-roam-dailies-directory "%<%Y-%m-%d>.org")
+     :if-new (file+head "%<%Y-%m-%d>.org"
-                         "#+title: %<%Y-%m-%d>\n")))
+                        "#+title: %<%Y-%m-%d>\n")))
   "Capture templates for daily-notes in Org-roam.
+Note that for daily files to show up in the calendar, they have to be of format
+\"org-time-string.org\".
 See `org-roam-capture-templates' for the template documentation."
   :group 'org-roam
   :type '(repeat
@@ -104,13 +105,13 @@ See `org-roam-capture-templates' for the template documentation."
                                                         (const :format "" file+olp)
                                                         (string :tag "  File")
                                                         (list :tag "Outline path"
-                                                              (repeat string :tag "Headline")))
+                                                              (repeat (string :tag "Headline"))))
                                                   (list :tag "File & Head Content & Outline path"
                                                         (const :format "" file+head+olp)
                                                         (string :tag "  File")
                                                         (string :tag "  Head Content")
                                                         (list :tag "Outline path"
-                                                              (repeat string :tag "Headline")))))
+                                                              (repeat (string :tag "Headline"))))))
                                          ((const :format "%v " :prepend) (const t))
                                          ((const :format "%v " :immediate-finish) (const t))
                                          ((const :format "%v " :jump-to-captured) (const t))
@@ -126,55 +127,23 @@ See `org-roam-capture-templates' for the template documentation."
                                          ((const :format "%v " :table-line-pos) (string))
                                          ((const :format "%v " :kill-buffer) (const t))))))))
 
-;;;; Utilities
+;;; Commands
-(defun org-roam-dailies-directory--get-absolute-path ()
+;;;; Today
-  "Get absolute path to `org-roam-dailies-directory'."
+;;;###autoload
-  (expand-file-name org-roam-dailies-directory org-roam-directory))
-
-(defun org-roam-dailies-find-directory ()
-  "Find and open `org-roam-dailies-directory'."
-  (interactive)
-  (find-file (org-roam-dailies-directory--get-absolute-path)))
-
-(defun org-roam-dailies--daily-note-p (&optional file)
-  "Return t if FILE is an Org-roam daily-note, nil otherwise.
-
-If FILE is not specified, use the current buffer's file-path."
-  (when-let ((path (or file
-                       (buffer-base-buffer (buffer-file-name))))
-             (directory (org-roam-dailies-directory--get-absolute-path)))
-    (setq path (expand-file-name path))
-    (save-match-data
-      (and
-       (org-roam--org-file-p path)
-       (f-descendant-of-p path directory)))))
-
-(defun org-roam-dailies--capture (time &optional goto)
-  "Capture an entry in a daily-note for TIME, creating it if necessary.
-
-When GOTO is non-nil, go the note without creating an entry."
-  (org-roam-capture- :goto (when goto '(4))
-                     :node (org-roam-node-create)
-                     :templates org-roam-dailies-capture-templates
-                     :props (list :default-time time)))
-
-;;;; Commands
-;;; Today
 (defun org-roam-dailies-capture-today (&optional goto)
   "Create an entry in the daily-note for today.
-
 When GOTO is non-nil, go the note without creating an entry."
   (interactive "P")
-  (org-roam-dailies--capture (current-time) goto)
+  (org-roam-dailies--capture (current-time) goto))
-  (when goto
-    (run-hooks 'org-roam-dailies-find-file-hook)))
 
-(defun org-roam-dailies-find-today ()
+;;;###autoload
+(defun org-roam-dailies-goto-today ()
   "Find the daily-note for today, creating it if necessary."
   (interactive)
   (org-roam-dailies-capture-today t))
 
-;;; Tomorrow
+;;;; Tomorrow
+;;;###autoload
 (defun org-roam-dailies-capture-tomorrow (n &optional goto)
   "Create an entry in the daily-note for tomorrow.
 
@@ -185,7 +154,8 @@ creating an entry."
   (interactive "p")
   (org-roam-dailies--capture (time-add (* n 86400) (current-time)) goto))
 
-(defun org-roam-dailies-find-tomorrow (n)
+;;;###autoload
+(defun org-roam-dailies-goto-tomorrow (n)
   "Find the daily-note for tomorrow, creating it if necessary.
 
 With numeric argument N, use the daily-note N days in the
@@ -193,7 +163,8 @@ future."
   (interactive "p")
   (org-roam-dailies-capture-tomorrow n t))
 
-;;; Yesterday
+;;;; Yesterday
+;;;###autoload
 (defun org-roam-dailies-capture-yesterday (n &optional goto)
   "Create an entry in the daily-note for yesteday.
 
@@ -203,7 +174,8 @@ When GOTO is non-nil, go the note without creating an entry."
   (interactive "p")
   (org-roam-dailies-capture-tomorrow (- n) goto))
 
-(defun org-roam-dailies-find-yesterday (n)
+;;;###autoload
+(defun org-roam-dailies-goto-yesterday (n)
   "Find the daily-note for yesterday, creating it if necessary.
 
 With numeric argument N, use the daily-note N days in the
@@ -211,85 +183,29 @@ future."
   (interactive "p")
   (org-roam-dailies-capture-tomorrow (- n) t))
 
-;;; Calendar
+;;;; Date
-(defvar org-roam-dailies-calendar-hook (list 'org-roam-dailies-calendar-mark-entries)
+;;;###autoload
-  "Hooks to run when showing the `org-roam-dailies-calendar'.")
-
-(defun org-roam-dailies-calendar--install-hook ()
-  "Install Org-roam-dailies hooks to calendar."
-  (add-hook 'calendar-today-visible-hook #'org-roam-dailies-calendar--run-hook)
-  (add-hook 'calendar-today-invisible-hook #'org-roam-dailies-calendar--run-hook))
-
-(defun org-roam-dailies-calendar--run-hook ()
-  "Run Org-roam-dailies hooks to calendar."
-  (run-hooks 'org-roam-dailies-calendar-hook)
-  (remove-hook 'calendar-today-visible-hook #'org-roam-dailies-calendar--run-hook)
-  (remove-hook 'calendar-today-invisible-hook #'org-roam-dailies-calendar--run-hook))
-
-(defun org-roam-dailies-calendar--file-to-date (&optional file)
-  "Convert FILE to date.
-
-Return (MONTH DAY YEAR)."
-  (let ((file (or file
-                  (buffer-base-buffer (buffer-file-name)))))
-    (cl-destructuring-bind (_ _ _ d m y _ _ _)
-        (org-parse-time-string
-         (file-name-sans-extension
-          (file-name-nondirectory file)))
-      (list m d y))))
-
-(defun org-roam-dailies-calendar--date-to-time (date)
-  "Convert DATE as returned from the calendar (MONTH DAY YEAR) to a time."
-  (encode-time 0 0 0 (nth 1 date) (nth 0 date) (nth 2 date)))
-
-(defun org-roam-dailies-calendar-mark-entries ()
-  "Mark days in the calendar for which a daily-note is present."
-  (when (file-exists-p (org-roam-dailies-directory--get-absolute-path))
-    (dolist (date (mapcar #'org-roam-dailies-calendar--file-to-date
-                          (org-roam-dailies--list-files)))
-      (when (calendar-date-is-visible-p date)
-        (calendar-mark-visible-date date 'org-roam-dailies-calendar-note)))))
-
-;;; Date
 (defun org-roam-dailies-capture-date (&optional goto prefer-future)
   "Create an entry in the daily-note for a date using the calendar.
-
 Prefer past dates, unless PREFER-FUTURE is non-nil.
-
 With a `C-u' prefix or when GOTO is non-nil, go the note without
 creating an entry."
   (interactive "P")
-  (org-roam-dailies-calendar--install-hook)
+  (let ((time (let ((org-read-date-prefer-future prefer-future))
-  (let* ((time-str (let ((org-read-date-prefer-future prefer-future))
+                (org-read-date t t nil (if goto
-                     (org-read-date nil nil nil (if goto
+                                           "Find daily-note: "
-                                                    "Find daily-note: "
+                                         "Capture to daily-note: ")))))
-                                                  "Capture to daily-note: "))))
+    (org-roam-dailies--capture time goto)))
-         (time (org-read-date nil t time-str)))
-    (org-roam-dailies--capture time goto)
-    (when goto
-      (run-hooks 'org-roam-dailies-find-file-hook))))
 
-(defun org-roam-dailies-find-date (&optional prefer-future)
+;;;###autoload
+(defun org-roam-dailies-goto-date (&optional prefer-future)
   "Find the daily-note for a date using the calendar, creating it if necessary.
 Prefer past dates, unless PREFER-FUTURE is non-nil."
   (interactive)
   (org-roam-dailies-capture-date t prefer-future))
 
-;;; Navigation
+;;;; Navigation
-(defun org-roam-dailies--list-files (&rest extra-files)
+(defun org-roam-dailies-goto-next-note (&optional n)
-  "List all files in `org-roam-dailies-directory'.
-EXTRA-FILES can be used to append extra files to the list."
-  (let ((dir (org-roam-dailies-directory--get-absolute-path))
-        (regexp (rx-to-string `(and "." (or ,@org-roam-file-extensions)))))
-    (append (--remove (let ((file (file-name-nondirectory it)))
-                        (when (or (auto-save-file-name-p file)
-                                  (backup-file-name-p file)
-                                  (string-match "^\\." file))
-                          it))
-                      (directory-files-recursively dir regexp))
-            extra-files)))
-
-(defun org-roam-dailies-find-next-note (&optional n)
   "Find next daily-note.
 
 With numeric argument N, find note N days in the future. If N is
@@ -317,28 +233,103 @@ negative, find note N days in the past."
     (find-file note)
     (run-hooks 'org-roam-dailies-find-file-hook)))
 
-(defun org-roam-dailies-find-previous-note (&optional n)
+(defun org-roam-dailies-goto-previous-note (&optional n)
   "Find previous daily-note.
 
 With numeric argument N, find note N days in the past. If N is
 negative, find note N days in the future."
   (interactive "p")
   (let ((n (if n (- n) -1)))
-    (org-roam-dailies-find-next-note n)))
+    (org-roam-dailies-goto-next-note n)))
 
-;;;; Bindings
+(defun org-roam-dailies--list-files (&rest extra-files)
+  "List all files in `org-roam-dailies-directory'.
+EXTRA-FILES can be used to append extra files to the list."
+  (let ((dir (expand-file-name org-roam-dailies-directory org-roam-directory))
+        (regexp (rx-to-string `(and "." (or ,@org-roam-file-extensions)))))
+    (append (--remove (let ((file (file-name-nondirectory it)))
+                        (when (or (auto-save-file-name-p file)
+                                  (backup-file-name-p file)
+                                  (string-match "^\\." file))
+                          it))
+                      (directory-files-recursively dir regexp))
+            extra-files)))
+
+(defun org-roam-dailies--daily-note-p (&optional file)
+  "Return t if FILE is an Org-roam daily-note, nil otherwise.
+If FILE is not specified, use the current buffer's file-path."
+  (when-let ((path (expand-file-name
+                    (or file
+                        (buffer-file-name (buffer-base-buffer)))))
+             (directory (expand-file-name org-roam-dailies-directory org-roam-directory)))
+    (setq path (expand-file-name path))
+    (save-match-data
+      (and
+       (org-roam-file-p path)
+       (f-descendant-of-p path directory)))))
+
+;;;###autoload
+(defun org-roam-dailies-find-directory ()
+  "Find and open `org-roam-dailies-directory'."
+  (interactive)
+  (find-file (expand-file-name org-roam-dailies-directory org-roam-directory)))
+
+;;; Calendar integration
+(defun org-roam-dailies-calendar--file-to-date (file)
+  "Convert FILE to date.
+Return (MONTH DAY YEAR) or nil if not an Org time-string."
+  (ignore-errors
+    (cl-destructuring-bind (_ _ _ d m y _ _ _)
+        (org-parse-time-string
+         (file-name-sans-extension
+          (file-name-nondirectory file)))
+      (list m d y))))
+
+(defun org-roam-dailies-calendar-mark-entries ()
+  "Mark days in the calendar for which a daily-note is present."
+  (when (file-exists-p (expand-file-name org-roam-dailies-directory org-roam-directory))
+    (dolist (date (remove nil
+                          (mapcar #'org-roam-dailies-calendar--file-to-date
+                                  (org-roam-dailies--list-files))))
+      (when (calendar-date-is-visible-p date)
+        (calendar-mark-visible-date date 'org-roam-dailies-calendar-note)))))
+
+(add-hook 'calendar-today-visible-hook #'org-roam-dailies-calendar-mark-entries)
+(add-hook 'calendar-today-invisible-hook #'org-roam-dailies-calendar-mark-entries)
+
+;;; Capture implementation
+(add-to-list 'org-roam-capture--template-keywords :override-default-time)
+
+(defun org-roam-dailies--capture (time &optional goto)
+  "Capture an entry in a daily-note for TIME, creating it if necessary.
+When GOTO is non-nil, go the note without creating an entry."
+  (let ((org-roam-directory (expand-file-name org-roam-dailies-directory org-roam-directory)))
+    (org-roam-capture- :goto (when goto '(4))
+                       :node (org-roam-node-create)
+                       :templates org-roam-dailies-capture-templates
+                       :props (list :override-default-time time)))
+  (when goto (run-hooks 'org-roam-dailies-find-file-hook)))
+
+(add-hook 'org-roam-capture-preface-hook #'org-roam-dailies--override-capture-time-h)
+(defun org-roam-dailies--override-capture-time-h ()
+  "Override the `:default-time' with the time from `:override-default-time'."
+  (prog1 nil
+    (when (org-roam-capture--get :override-default-time)
+      (org-capture-put :default-time (org-roam-capture--get :override-default-time)))))
+
+;;; Bindings
 (defvar org-roam-dailies-map (make-sparse-keymap)
   "Keymap for `org-roam-dailies'.")
 
 (define-prefix-command 'org-roam-dailies-map)
 
-(define-key org-roam-dailies-map (kbd "d") #'org-roam-dailies-find-today)
+(define-key org-roam-dailies-map (kbd "d") #'org-roam-dailies-goto-today)
-(define-key org-roam-dailies-map (kbd "y") #'org-roam-dailies-find-yesterday)
+(define-key org-roam-dailies-map (kbd "y") #'org-roam-dailies-goto-yesterday)
-(define-key org-roam-dailies-map (kbd "t") #'org-roam-dailies-find-tomorrow)
+(define-key org-roam-dailies-map (kbd "t") #'org-roam-dailies-goto-tomorrow)
 (define-key org-roam-dailies-map (kbd "n") #'org-roam-dailies-capture-today)
-(define-key org-roam-dailies-map (kbd "f") #'org-roam-dailies-find-next-note)
+(define-key org-roam-dailies-map (kbd "f") #'org-roam-dailies-goto-next-note)
-(define-key org-roam-dailies-map (kbd "b") #'org-roam-dailies-find-previous-note)
+(define-key org-roam-dailies-map (kbd "b") #'org-roam-dailies-goto-previous-note)
-(define-key org-roam-dailies-map (kbd "c") #'org-roam-dailies-find-date)
+(define-key org-roam-dailies-map (kbd "c") #'org-roam-dailies-goto-date)
 (define-key org-roam-dailies-map (kbd "v") #'org-roam-dailies-capture-date)
 (define-key org-roam-dailies-map (kbd ".") #'org-roam-dailies-find-directory)
 

extensions/org-roam-graph.el

@@ -0,0 +1,283 @@
+;;; org-roam-graph.el --- Basic graphing functionality for Org-roam -*- coding: utf-8; lexical-binding: t; -*-
+
+;; Copyright © 2020-2021 Jethro Kuan <jethrokuan95@gmail.com>
+
+;; Author: Jethro Kuan <jethrokuan95@gmail.com>
+;; URL: https://github.com/org-roam/org-roam
+;; Keywords: org-mode, roam, convenience
+;; Version: 2.1.0
+;; Package-Requires: ((emacs "26.1") (org "9.4") (org-roam "2.1"))
+
+;; This file is NOT part of GNU Emacs.
+
+;; This program is free software; you can redistribute it and/or modify
+;; it under the terms of the GNU General Public License as published by
+;; the Free Software Foundation; either version 3, or (at your option)
+;; any later version.
+;;
+;; This program is distributed in the hope that it will be useful,
+;; but WITHOUT ANY WARRANTY; without even the implied warranty of
+;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+;; GNU General Public License for more details.
+;;
+;; You should have received a copy of the GNU General Public License
+;; along with GNU Emacs; see the file COPYING.  If not, write to the
+;; Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor,
+;; Boston, MA 02110-1301, USA.
+
+;;; Commentary:
+;;
+;; This extension implements capability to build and generate graphs in Org-roam
+;; with the help of Graphviz.
+;;
+;;; Code:
+(require 'xml) ;xml-escape-string
+(require 'org-roam)
+
+;;; Options
+(defcustom org-roam-graph-viewer (executable-find "firefox")
+  "Method to view the org-roam graph.
+It may be one of the following:
+  - a string representing the path to the executable for viewing the graph.
+  - a function accepting a single argument: the graph file path.
+  - nil uses `view-file' to view the graph."
+  :type '(choice
+          (string   :tag "Path to executable")
+          (function :tag "Function to display graph" eww-open-file)
+          (const    :tag "view-file"))
+  :group 'org-roam)
+
+(defcustom org-roam-graph-executable "dot"
+  "Path to graphing executable, or its name."
+  :type 'string
+  :group 'org-roam)
+
+(defcustom org-roam-graph-filetype "svg"
+  "File type to generate when producing graphs."
+  :type 'string
+  :group 'org-roam)
+
+
+(defcustom org-roam-graph-extra-config nil
+  "Extra options passed to graphviz.
+Example:
+ '((\"rankdir\" . \"LR\"))"
+  :type '(alist)
+  :group 'org-roam)
+
+(defcustom org-roam-graph-edge-extra-config nil
+  "Extra edge options passed to graphviz.
+Example:
+ '((\"dir\" . \"back\"))"
+  :type '(alist)
+  :group 'org-roam)
+
+(defcustom org-roam-graph-node-extra-config
+  '(("id" . (("style"      . "bold,rounded,filled")
+             ("fillcolor"  . "#EEEEEE")
+             ("color"      . "#C9C9C9")
+             ("fontcolor"  . "#111111")))
+    ("http" . (("style"      . "rounded,filled")
+               ("fillcolor"  . "#EEEEEE")
+               ("color"      . "#C9C9C9")
+               ("fontcolor"  . "#0A97A6")))
+    ("https" . (("shape"      . "rounded,filled")
+                ("fillcolor"  . "#EEEEEE")
+                ("color"      . "#C9C9C9")
+                ("fontcolor"  . "#0A97A6"))))
+  "Extra options for graphviz nodes."
+  :type '(alist)
+  :group 'org-roam)
+
+(defcustom org-roam-graph-link-hidden-types
+  '("file")
+  "What sort of links to hide from the Org-roam graph."
+  :type '(repeat string)
+  :group 'org-roam)
+
+(defcustom org-roam-graph-max-title-length 100
+  "Maximum length of titles in graph nodes."
+  :type 'number
+  :group 'org-roam)
+
+(defcustom org-roam-graph-shorten-titles 'truncate
+  "Determines how long titles appear in graph nodes.
+Recognized values are the symbols `truncate' and `wrap', in which
+cases the title will be truncated or wrapped, respectively, if it
+is longer than `org-roam-graph-max-title-length'.
+
+All other values including nil will have no effect."
+  :type '(choice
+          (const :tag "truncate" truncate)
+          (const :tag "wrap" wrap)
+          (const :tag "no" nil))
+  :group 'org-roam)
+
+;;; Interactive command
+;;;###autoload
+(defun org-roam-graph (&optional arg node)
+  "Build and possibly display a graph for NODE.
+ARG may be any of the following values:
+  - nil       show the graph.
+  - `\\[universal-argument]'     show the graph for NODE.
+  - `\\[universal-argument]' N   show the graph for NODE limiting nodes to N steps."
+  (interactive
+   (list current-prefix-arg
+         (and current-prefix-arg
+              (org-roam-node-at-point 'assert))))
+  (let ((graph (cl-typecase arg
+                 (null (org-roam-graph--dot nil 'all-nodes))
+                 (cons (org-roam-graph--dot (org-roam-graph--connected-component
+                                             (org-roam-node-id node) 0)))
+                 (integer (org-roam-graph--dot (org-roam-graph--connected-component
+                                                (org-roam-node-id node) (abs arg)))))))
+    (org-roam-graph--build graph #'org-roam-graph--open)))
+
+;;; Generation and Build process
+(defun org-roam-graph--build (graph &optional callback)
+  "Generate the GRAPH, and execute CALLBACK when process exits successfully.
+CALLBACK is passed the graph file as its sole argument."
+  (unless (stringp org-roam-graph-executable)
+    (user-error "`org-roam-graph-executable' is not a string"))
+  (unless (executable-find org-roam-graph-executable)
+    (user-error (concat "Cannot find executable \"%s\" to generate the graph.  "
+                        "Please adjust `org-roam-graph-executable'")
+                org-roam-graph-executable))
+  (let* ((temp-dot   (make-temp-file "graph." nil ".dot" graph))
+         (temp-graph (make-temp-file "graph." nil (concat "." org-roam-graph-filetype))))
+    (org-roam-message "building graph")
+    (make-process
+     :name "*org-roam-graph--build-process*"
+     :buffer "*org-roam-graph--build-process*"
+     :command `(,org-roam-graph-executable ,temp-dot "-T" ,org-roam-graph-filetype "-o" ,temp-graph)
+     :sentinel (when callback
+                 (lambda (process _event)
+                   (when (= 0 (process-exit-status process))
+                     (funcall callback temp-graph)))))))
+
+(defun org-roam-graph--dot (&optional edges all-nodes)
+  "Build the graphviz given the EDGES of the graph.
+If ALL-NODES, include also nodes without edges."
+  (let ((org-roam-directory-temp org-roam-directory)
+        (nodes-table (make-hash-table :test #'equal))
+        (seen-nodes (list))
+        (edges (or edges (org-roam-db-query [:select :distinct [source dest type] :from links]))))
+    (pcase-dolist (`(,id ,file ,title)
+                   (org-roam-db-query [:select [id file title] :from nodes]))
+      (puthash id (org-roam-node-create :file file :id id :title title) nodes-table))
+    (with-temp-buffer
+      (setq-local org-roam-directory org-roam-directory-temp)
+      (insert "digraph \"org-roam\" {\n")
+      (dolist (option org-roam-graph-extra-config)
+        (insert (org-roam-graph--dot-option option) ";\n"))
+      (insert (format " edge [%s];\n"
+                      (mapconcat (lambda (var)
+                                   (org-roam-graph--dot-option var nil "\""))
+                                 org-roam-graph-edge-extra-config
+                                 ",")))
+      (pcase-dolist (`(,source ,dest ,type) edges)
+        (unless (member type org-roam-graph-link-hidden-types)
+          (pcase-dolist (`(,node ,node-type) `((,source "id")
+                                               (,dest ,type)))
+            (unless (member node seen-nodes)
+              (insert (org-roam-graph--format-node
+                       (or (gethash node nodes-table) node) node-type))
+              (push node seen-nodes)))
+          (insert (format "  \"%s\" -> \"%s\";\n"
+                          (xml-escape-string source)
+                          (xml-escape-string dest)))))
+      (when all-nodes
+        (maphash (lambda (id node)
+                   (unless (member id seen-nodes)
+                     (insert (org-roam-graph--format-node node "id"))))
+                 nodes-table))
+      (insert "}")
+      (buffer-string))))
+
+(defun org-roam-graph--connected-component (id distance)
+  "Return the edges for all nodes reachable from/connected to ID.
+DISTANCE is the maximum distance away from the root node."
+  (let* ((query
+          (if (= distance 0)
+              "
+WITH RECURSIVE
+  links_of(source, dest) AS
+  (SELECT source, dest FROM links UNION
+   SELECT dest, source FROM links),
+   connected_component(source) AS
+  (SELECT dest FROM links_of WHERE source = $s1 UNION
+   SELECT dest FROM links_of JOIN connected_component USING(source))
+SELECT DISTINCT source, dest, type FROM links
+WHERE source IN connected_component OR dest IN connected_component;"
+            "
+WITH RECURSIVE
+  links_of(source, dest) AS
+  (SELECT source, dest FROM links UNION
+   SELECT dest, source FROM links),
+  connected_component(source, trace) AS
+  (VALUES ($s1 , json_array($s1)) UNION
+   SELECT lo.dest, json_insert(cc.trace, '$[' || json_array_length(cc.trace) || ']', lo.dest) FROM
+   connected_component AS cc JOIN links_of AS lo USING(source)
+   WHERE (
+    -- Avoid cycles by only visiting each node once.
+    (SELECT count(*) FROM json_each(cc.trace) WHERE json_each.value == lo.dest) == 0
+    -- Note: BFS is cut off early here.
+    AND json_array_length(cc.trace) < $s2)),
+  nodes(source) as (SELECT DISTINCT source
+   FROM connected_component GROUP BY source ORDER BY min(json_array_length(trace)))
+SELECT DISTINCT source, dest, type FROM links WHERE source IN nodes OR dest IN nodes;")))
+    (org-roam-db-query query id distance)))
+
+(defun org-roam-graph--dot-option (option &optional wrap-key wrap-val)
+  "Return dot string of form KEY=VAL for OPTION cons.
+If WRAP-KEY is non-nil it wraps the KEY.
+If WRAP-VAL is non-nil it wraps the VAL."
+  (concat wrap-key (car option) wrap-key
+          "="
+          wrap-val (cdr option) wrap-val))
+
+(defun org-roam-graph--format-node (node type)
+  "Return a graphviz NODE with TYPE.
+Handles both Org-roam nodes, and string nodes (e.g. urls)."
+  (let (node-id node-properties)
+    (if (org-roam-node-p node)
+        (let* ((title (org-roam-quote-string (org-roam-node-title node)))
+               (shortened-title
+                (org-roam-quote-string
+                 (pcase org-roam-graph-shorten-titles
+                   (`truncate (truncate-string-to-width title org-roam-graph-max-title-length nil nil "..."))
+                   (`wrap (s-word-wrap org-roam-graph-max-title-length title))
+                   (_ title)))))
+          (setq node-id (org-roam-node-id node)
+                node-properties `(("label"   . ,shortened-title)
+                                  ("URL"     . ,(concat "org-protocol://roam-node?node="
+                                                        (url-hexify-string (org-roam-node-id node))))
+                                  ("tooltip" . ,(xml-escape-string title)))))
+      (setq node-id node
+            node-properties (append `(("label" . ,(concat type ":" node)))
+                                    (when (member type (list "http" "https"))
+                                      `(("URL" . ,(xml-escape-string (concat type ":" node))))))))
+    (format "\"%s\" [%s];\n"
+            node-id
+            (mapconcat (lambda (n)
+                         (org-roam-graph--dot-option n nil "\""))
+                       (append (cdr (assoc type org-roam-graph-node-extra-config))
+                               node-properties) ","))))
+
+(defun org-roam-graph--open (file)
+  "Open FILE using `org-roam-graph-viewer' with `view-file' as a fallback."
+  (pcase org-roam-graph-viewer
+    ((pred stringp)
+     (if (executable-find org-roam-graph-viewer)
+         (condition-case err
+             (call-process org-roam-graph-viewer nil 0 nil file)
+           (error (user-error "Failed to open org-roam graph: %s" err)))
+       (user-error "Executable not found: \"%s\"" org-roam-graph-viewer)))
+    ((pred functionp) (funcall org-roam-graph-viewer file))
+    ('nil (view-file file))
+    (_ (signal 'wrong-type-argument `((functionp stringp null) ,org-roam-graph-viewer)))))
+
+
+(provide 'org-roam-graph)
+
+;;; org-roam-graph.el ends here

extensions/org-roam-overlay.el

@@ -0,0 +1,96 @@
+;;; org-roam-overlay.el --- Link overlay for [id:] links to Org-roam nodes -*- coding: utf-8; lexical-binding: t; -*-
+
+;; Copyright © 2020-2021 Jethro Kuan <jethrokuan95@gmail.com>
+
+;; Author: Jethro Kuan <jethrokuan95@gmail.com>
+;; URL: https://github.com/org-roam/org-roam
+;; Keywords: org-mode, roam, convenience
+;; Version: 2.1.0
+;; Package-Requires: ((emacs "26.1") (org "9.4") (org-roam "2.1"))
+
+;; This file is NOT part of GNU Emacs.
+
+;; This program is free software; you can redistribute it and/or modify
+;; it under the terms of the GNU General Public License as published by
+;; the Free Software Foundation; either version 3, or (at your option)
+;; any later version.
+;;
+;; This program is distributed in the hope that it will be useful,
+;; but WITHOUT ANY WARRANTY; without even the implied warranty of
+;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+;; GNU General Public License for more details.
+;;
+;; You should have received a copy of the GNU General Public License
+;; along with GNU Emacs; see the file COPYING.  If not, write to the
+;; Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor,
+;; Boston, MA 02110-1301, USA.
+
+;;; Commentary:
+;;
+;; This extension provides allows to render [[id:]] links that don't have an
+;; asscoiated descriptor with an overlay that displays the node's current title.
+;;
+;;; Code:
+(require 'org-roam)
+
+(defface org-roam-overlay
+  '((((class color) (background light))
+     :background "grey90" :box (:line-width -1 :color "black"))
+    (((class color) (background dark))
+     :background "grey10" :box (:line-width -1 :color "white")))
+  "Face for the Org-roam overlay."
+  :group 'org-roam-faces)
+
+(defun org-roam-overlay--make (l r &rest props)
+  "Make an overlay from L to R with PROPS."
+  (let ((o (make-overlay l (or r l))))
+    (overlay-put o 'category 'org-roam)
+    (while props (overlay-put o (pop props) (pop props)))
+    o))
+
+(defun org-roam-overlay-make-link-overlay (link)
+  "Create overlay for LINK."
+  (save-excursion
+    (save-match-data
+      (let* ((type (org-element-property :type link))
+             (id (org-element-property :path link))
+             (pos (org-element-property :end link))
+             (desc-p (org-element-property :contents-begin link))
+             node)
+        (when (and (string-equal type "id")
+                   (setq node (org-roam-node-from-id id))
+                   (not desc-p))
+          (org-roam-overlay--make
+           pos pos
+           'after-string (format "%s "
+                                 (propertize (org-roam-node-title node)
+                                             'face 'org-roam-overlay))))))))
+
+(defun org-roam-overlay-enable ()
+  "Enable Org-roam overlays."
+  (org-roam-db-map-links
+   (list #'org-roam-overlay-make-link-overlay)))
+
+(defun org-roam-overlay-disable ()
+  "Disable Org-roam overlays."
+  (remove-overlays nil nil 'category 'org-roam))
+
+(defun org-roam-overlay-redisplay ()
+  "Redisplay Org-roam overlays."
+  (org-roam-overlay-disable)
+  (org-roam-overlay-enable))
+
+(define-minor-mode org-roam-overlay-mode
+  "Overlays for Org-roam ID links.
+Org-roam overlay mode is a minor mode.  When enabled,
+overlay displaying the node's title is displayed."
+  :lighter " org-roam-overlay"
+  (if org-roam-overlay-mode
+      (progn
+        (org-roam-overlay-enable)
+        (add-hook 'after-save-hook #'org-roam-overlay-redisplay nil t))
+    (org-roam-overlay-disable)
+    (remove-hook 'after-save-hook #'org-roam-overlay-redisplay t)))
+
+(provide 'org-roam-overlay)
+;;; org-roam-overlay.el ends here

extensions/org-roam-protocol.el

@@ -0,0 +1,192 @@
+;;; org-roam-protocol.el --- Protocol handler for roam:// links  -*- coding: utf-8; lexical-binding: t; -*-
+
+;; Copyright © 2020-2021 Jethro Kuan <jethrokuan95@gmail.com>
+;; Author: Jethro Kuan <jethrokuan95@gmail.com>
+;; URL: https://github.com/org-roam/org-roam
+;; Keywords: org-mode, roam, convenience
+;; Version: 2.1.0
+;; Package-Requires: ((emacs "26.1") (org "9.4") (org-roam "2.1"))
+
+;; This file is NOT part of GNU Emacs.
+
+;; This program is free software; you can redistribute it and/or modify
+;; it under the terms of the GNU General Public License as published by
+;; the Free Software Foundation; either version 3, or (at your option)
+;; any later version.
+;;
+;; This program is distributed in the hope that it will be useful,
+;; but WITHOUT ANY WARRANTY; without even the implied warranty of
+;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+;; GNU General Public License for more details.
+;;
+;; You should have received a copy of the GNU General Public License
+;; along with GNU Emacs; see the file COPYING.  If not, write to the
+;; Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor,
+;; Boston, MA 02110-1301, USA.
+
+;;; Commentary:
+;;
+;; This extension extends `org-protocol', adding custom Org-roam handlers to it
+;; to provide the next new protocols:
+;;
+;; 1. "roam-node": This protocol simply opens the node given by the node ID
+;; 2. "roam-ref": This protocol creates or opens the node with the given REF
+;;
+;; You can find detailed instructions on how to setup the protocol in the
+;; documentation for Org-roam.
+;;
+;;; Code:
+(require 'org-protocol)
+(require 'ol) ;; for org-link-decode
+(require 'org-roam)
+
+;;; Options
+(defcustom org-roam-protocol-store-links nil
+  "Whether to store links when capturing websites with `org-roam-protocol'."
+  :type 'boolean
+  :group 'org-roam)
+
+(defcustom org-roam-capture-ref-templates
+  '(("r" "ref" plain "%?"
+     :if-new (file+head "${slug}.org"
+                        "#+title: ${title}")
+     :unnarrowed t))
+  "The Org-roam templates used during a capture from the roam-ref protocol.
+See `org-roam-capture-templates' for the template documentation."
+  :group 'org-roam
+  :type '(repeat
+          (choice (list :tag "Multikey description"
+                        (string :tag "Keys       ")
+                        (string :tag "Description"))
+                  (list :tag "Template entry"
+                        (string :tag "Keys           ")
+                        (string :tag "Description    ")
+                        (choice :tag "Capture Type   " :value entry
+                                (const :tag "Org entry" entry)
+                                (const :tag "Plain list item" item)
+                                (const :tag "Checkbox item" checkitem)
+                                (const :tag "Plain text" plain)
+                                (const :tag "Table line" table-line))
+                        (choice :tag "Template       "
+                                (string)
+                                (list :tag "File"
+                                      (const :format "" file)
+                                      (file :tag "Template file"))
+                                (list :tag "Function"
+                                      (const :format "" function)
+                                      (function :tag "Template function")))
+                        (plist :inline t
+                               ;; Give the most common options as checkboxes
+                               :options (((const :format "%v " :if-new)
+                                          (choice :tag "Node location"
+                                                  (list :tag "File"
+                                                        (const :format "" file)
+                                                        (string :tag "  File"))
+                                                  (list :tag "File & Head Content"
+                                                        (const :format "" file+head)
+                                                        (string :tag "  File")
+                                                        (string :tag "  Head Content"))
+                                                  (list :tag "File & Outline path"
+                                                        (const :format "" file+olp)
+                                                        (string :tag "  File")
+                                                        (list :tag "Outline path"
+                                                              (repeat (string :tag "Headline"))))
+                                                  (list :tag "File & Head Content & Outline path"
+                                                        (const :format "" file+head+olp)
+                                                        (string :tag "  File")
+                                                        (string :tag "  Head Content")
+                                                        (list :tag "Outline path"
+                                                              (repeat (string :tag "Headline"))))))
+                                         ((const :format "%v " :prepend) (const t))
+                                         ((const :format "%v " :immediate-finish) (const t))
+                                         ((const :format "%v " :jump-to-captured) (const t))
+                                         ((const :format "%v " :empty-lines) (const 1))
+                                         ((const :format "%v " :empty-lines-before) (const 1))
+                                         ((const :format "%v " :empty-lines-after) (const 1))
+                                         ((const :format "%v " :clock-in) (const t))
+                                         ((const :format "%v " :clock-keep) (const t))
+                                         ((const :format "%v " :clock-resume) (const t))
+                                         ((const :format "%v " :time-prompt) (const t))
+                                         ((const :format "%v " :tree-type) (const week))
+                                         ((const :format "%v " :unnarrowed) (const t))
+                                         ((const :format "%v " :table-line-pos) (string))
+                                         ((const :format "%v " :kill-buffer) (const t))))))))
+
+;;; Handlers
+(defun org-roam-protocol-open-ref (info)
+  "Process an org-protocol://roam-ref?ref= style url with INFO.
+
+It opens or creates a note with the given ref.
+
+  javascript:location.href = \\='org-protocol://roam-ref?template=r&ref=\\='+ \\
+        encodeURIComponent(location.href) + \\='&title=\\=' + \\
+        encodeURIComponent(document.title) + \\='&body=\\=' + \\
+        encodeURIComponent(window.getSelection())"
+  (unless (plist-get info :ref)
+    (user-error "No ref key provided"))
+  (org-roam-plist-map! (lambda (k v)
+                         (org-link-decode
+                          (if (equal k :ref)
+                              (org-protocol-sanitize-uri v)
+                            v))) info)
+  (when org-roam-protocol-store-links
+    (push (list (plist-get info :ref)
+                (plist-get info :title)) org-stored-links))
+  (org-link-store-props :type (and (string-match org-link-plain-re
+                                                 (plist-get info :ref))
+                                   (match-string 1 (plist-get info :ref)))
+                        :link (plist-get info :ref)
+                        :annotation (org-link-make-string (plist-get info :ref)
+                                                          (or (plist-get info :title)
+                                                              (plist-get info :ref)))
+                        :initial (or (plist-get info :body) ""))
+  (raise-frame)
+  (org-roam-capture-
+   :keys (plist-get info :template)
+   :node (org-roam-node-create :title (plist-get info :title))
+   :info (list :ref (plist-get info :ref)
+               :body (plist-get info :body))
+   :templates org-roam-capture-ref-templates)
+  nil)
+
+(defun org-roam-protocol-open-node (info)
+  "This handler simply opens the file with emacsclient.
+
+INFO is a plist containing additional information passed by the protocol URL.
+It should contain the FILE key, pointing to the path of the file to open.
+
+  Example protocol string:
+
+org-protocol://roam-node?node=uuid"
+  (when-let ((node (plist-get info :node)))
+    (raise-frame)
+    (org-roam-node-visit (org-roam-populate (org-roam-node-create :id node)) nil 'force))
+  nil)
+
+(push '("org-roam-ref"  :protocol "roam-ref"   :function org-roam-protocol-open-ref)
+      org-protocol-protocol-alist)
+(push '("org-roam-node"  :protocol "roam-node"   :function org-roam-protocol-open-node)
+      org-protocol-protocol-alist)
+
+;;; Capture implementation
+(add-hook 'org-roam-capture-preface-hook #'org-roam-protocol--try-capture-to-ref-h)
+(defun org-roam-protocol--try-capture-to-ref-h ()
+  "Try to capture to an existing node that match the ref."
+  (when-let ((node (and (plist-get org-roam-capture--info :ref)
+                        (org-roam-node-from-ref
+                         (plist-get org-roam-capture--info :ref)))))
+    (set-buffer (org-capture-target-buffer (org-roam-node-file node)))
+    (goto-char (org-roam-node-point node))
+    (widen)
+    (org-roam-node-id node)))
+
+(add-hook 'org-roam-capture-new-node-hook #'org-roam-protocol--insert-captured-ref-h)
+(defun org-roam-protocol--insert-captured-ref-h ()
+  "Insert the ref if any."
+  (when-let ((ref (plist-get org-roam-capture--info :ref)))
+    (org-roam-ref-add ref)))
+
+
+(provide 'org-roam-protocol)
+
+;;; org-roam-protocol.el ends here

org-roam-capture.el

@@ -1,12 +1,12 @@
 ;;; org-roam-capture.el --- Capture functionality -*- coding: utf-8; lexical-binding: t; -*-
 
-;; Copyright © 2020 Jethro Kuan <jethrokuan95@gmail.com>
+;; Copyright © 2020-2021 Jethro Kuan <jethrokuan95@gmail.com>
 
 ;; Author: Jethro Kuan <jethrokuan95@gmail.com>
 ;; URL: https://github.com/org-roam/org-roam
 ;; Keywords: org-mode, roam, convenience
-;; Version: 2.0.0
+;; Version: 2.1.0
-;; Package-Requires: ((emacs "26.1") (dash "2.13") (f "0.17.2") (s "1.12.0") (org "9.4") (emacsql "3.0.0") (emacsql-sqlite3 "1.0.2") (magit-section "2.90.1"))
+;; Package-Requires: ((emacs "26.1") (dash "2.13") (f "0.17.2") (org "9.4") (emacsql "3.0.0") (emacsql-sqlite "1.0.0") (magit-section "2.90.1"))
 
 ;; This file is NOT part of GNU Emacs.
 
@@ -27,35 +27,17 @@
 
 ;;; Commentary:
 ;;
-;; This library provides capture functionality for org-roam
+;; This module provides `org-capture' functionality for Org-roam. With this
+;; module the user can capture new nodes or capture new content to existing
+;; nodes.
+;;
 ;;; Code:
-;;;
+(require 'org-roam)
-;;;; Library Requires
-(require 'org-capture)
-(eval-when-compile
-  (require 'org-roam-macs))
-(require 'org-roam-db)
-(require 'dash)
-(require 'cl-lib)
 
-;; Declarations
+;;;; Declarations
-(declare-function org-roam-ref-add "org-roam" (ref))
+(defvar org-end-time-was-given)
-
-(defvar org-roam-directory)
-
-(defvar org-roam-capture--node nil
-  "The node passed during an Org-roam capture.
-This variable is populated dynamically, and is only non-nil
-during the Org-roam capture process.")
-
-(defvar org-roam-capture--info nil
-  "A property-list of additional information passed to the Org-roam template.
-This variable is populated dynamically, and is only non-nil
-during the Org-roam capture process.")
-
-(defconst org-roam-capture--template-keywords '(:if-new :id :link-description :call-location)
-  "Keywords used in `org-roam-capture-templates' specific to Org-roam.")
 
+;;; Options
 (defcustom org-roam-capture-templates
   '(("d" "default" plain "%?"
      :if-new (file+head "%<%Y%m%d%H%M%S>-${slug}.org"
@@ -121,15 +103,24 @@ the following options:
        The file will be created, prescribed an ID, and head content will be
        inserted into the file.
 
-   (file+olp \"path/to/file\" '(\"h1\" \"h2\"))
+   (file+olp \"path/to/file\" (\"h1\" \"h2\"))
        The file will be created, prescribed an ID. The OLP (h1, h2) will be
        created, and the point placed after.
 
-   (file+head+olp \"path/to/file\" \"head content\" '(\"h1\" \"h2\"))
+   (file+head+olp \"path/to/file\" \"head content\" (\"h1\" \"h2\"))
        The file will be created, prescribed an ID. Head content will be
        inserted at the start of the file. The OLP (h1, h2) will be created,
        and the point placed after.
 
+   (file+datetree \"path/to/file\" day)
+       The file will be created, prescribed an ID. Head content will be
+       inserted at the start of the file. The datetree will be created,
+       available options are day, week, month.
+
+   (node \"title or alias or ID of an existing node\")
+       The point will be placed for an existing node, based on either, its
+       title, alias or ID.
+
 The rest of the entry is a property list of additional options.  Recognized
 properties are:
 
@@ -311,13 +302,13 @@ streamlined user experience in Org-roam."
                                                         (const :format "" file+olp)
                                                         (string :tag "  File")
                                                         (list :tag "Outline path"
-                                                              (repeat string :tag "Headline")))
+                                                              (repeat (string :tag "Headline"))))
                                                   (list :tag "File & Head Content & Outline path"
                                                         (const :format "" file+head+olp)
                                                         (string :tag "  File")
                                                         (string :tag "  Head Content")
                                                         (list :tag "Outline path"
-                                                              (repeat string :tag "Headline")))))
+                                                              (repeat (string :tag "Headline"))))))
                                          ((const :format "%v " :prepend) (const t))
                                          ((const :format "%v " :immediate-finish) (const t))
                                          ((const :format "%v " :jump-to-captured) (const t))
@@ -333,77 +324,104 @@ streamlined user experience in Org-roam."
                                          ((const :format "%v " :table-line-pos) (string))
                                          ((const :format "%v " :kill-buffer) (const t))))))))
 
-(defvar org-roam-capture-new-node-hook (list #'org-roam-capture--insert-ref)
+(defcustom org-roam-capture-new-node-hook nil
   "Normal-mode hooks run when a new Org-roam node is created.
 The current point is the point of the new node.
-The hooks must not move the point.")
+The hooks must not move the point."
-
-(defcustom org-roam-capture-ref-templates
-  '(("r" "ref" plain "%?"
-     :if-new (file+head "${slug}.org"
-                        "#+title: ${title}")
-     :unnarrowed t))
-  "The Org-roam templates used during a capture from the roam-ref protocol.
-See `org-roam-capture-templates' for the template documentation."
   :group 'org-roam
-  :type '(repeat
+  :type 'hook)
-          (choice (list :tag "Multikey description"
-                        (string :tag "Keys       ")
-                        (string :tag "Description"))
-                  (list :tag "Template entry"
-                        (string :tag "Keys           ")
-                        (string :tag "Description    ")
-                        (choice :tag "Capture Type   " :value entry
-                                (const :tag "Org entry" entry)
-                                (const :tag "Plain list item" item)
-                                (const :tag "Checkbox item" checkitem)
-                                (const :tag "Plain text" plain)
-                                (const :tag "Table line" table-line))
-                        (choice :tag "Template       "
-                                (string)
-                                (list :tag "File"
-                                      (const :format "" file)
-                                      (file :tag "Template file"))
-                                (list :tag "Function"
-                                      (const :format "" function)
-                                      (function :tag "Template function")))
-                        (plist :inline t
-                               ;; Give the most common options as checkboxes
-                               :options (((const :format "%v " :if-new)
-                                          (choice :tag "Node location"
-                                                  (list :tag "File"
-                                                        (const :format "" file)
-                                                        (string :tag "  File"))
-                                                  (list :tag "File & Head Content"
-                                                        (const :format "" file+head)
-                                                        (string :tag "  File")
-                                                        (string :tag "  Head Content"))
-                                                  (list :tag "File & Outline path"
-                                                        (const :format "" file+olp)
-                                                        (string :tag "  File")
-                                                        (list :tag "Outline path"
-                                                              (repeat string :tag "Headline")))
-                                                  (list :tag "File & Head Content & Outline path"
-                                                        (const :format "" file+head+olp)
-                                                        (string :tag "  File")
-                                                        (string :tag "  Head Content")
-                                                        (list :tag "Outline path"
-                                                              (repeat string :tag "Headline")))))
-                                         ((const :format "%v " :prepend) (const t))
-                                         ((const :format "%v " :immediate-finish) (const t))
-                                         ((const :format "%v " :jump-to-captured) (const t))
-                                         ((const :format "%v " :empty-lines) (const 1))
-                                         ((const :format "%v " :empty-lines-before) (const 1))
-                                         ((const :format "%v " :empty-lines-after) (const 1))
-                                         ((const :format "%v " :clock-in) (const t))
-                                         ((const :format "%v " :clock-keep) (const t))
-                                         ((const :format "%v " :clock-resume) (const t))
-                                         ((const :format "%v " :time-prompt) (const t))
-                                         ((const :format "%v " :tree-type) (const week))
-                                         ((const :format "%v " :unnarrowed) (const t))
-                                         ((const :format "%v " :table-line-pos) (string))
-                                         ((const :format "%v " :kill-buffer) (const t))))))))
 
+(defvar org-roam-capture-preface-hook nil
+  "Hook run when Org-roam tries to determine capture location of the node.
+If any hook returns a value (which should be an ID), all hooks
+after it are ignored.
+
+With this hook you can hijack controls over the location of the
+node for which the capture process is currently running for, or
+use to just perform an arbitrary side effect, e.g. modify the
+state related to the capture process. See `org-roam-protocol' and
+`org-roam-dailies' as examples for what and how this hook is used
+for.
+
+If you're trying to perform the hijack, it's mandatory for you to:
+  1. Set the currently active buffer for editing operations using
+     `org-capture-target-buffer'.
+  2. Place the point in this buffer from where the location starts
+     from (e.g. if it's a file based node it should be the BOB,
+     otherwise it should be the position from where the heading
+     based node starts from).
+  3. Return the ID (as a string) of the capturing node.
+
+If you use this hook for any other purpose, but not the hijack,
+it's mandatory that you should return nil as the return value; so
+the capture process would be able to setup the capture buffer.
+
+If you need to do something when you capture new nodes, use
+`org-roam-capture-new-node-hook' instead of this hook.
+
+WARNING: This hook is primarily designed for the usage by the
+extensions and packages, and requires understanding of the
+internal capture process. If you don't understand it, you should
+learn these internals before using this or use it at your own
+risk breaking things.")
+
+;;; Variables
+
+(defvar org-roam-capture--node nil
+  "The node passed during an Org-roam capture.
+This variable is populated dynamically, and is only non-nil
+during the Org-roam capture process.")
+
+(defvar org-roam-capture--info nil
+  "A property-list of additional information passed to the Org-roam template.
+This variable is populated dynamically, and is only non-nil
+during the Org-roam capture process.")
+
+(defconst org-roam-capture--template-keywords (list :if-new :id :link-description :call-location
+                                                    :region)
+  "Keywords used in `org-roam-capture-templates' specific to Org-roam.")
+
+;;; Main entry point
+;;;###autoload
+(cl-defun org-roam-capture- (&key goto keys node info props templates)
+  "Main entry point of `org-roam-capture' module.
+GOTO and KEYS correspond to `org-capture' arguments.
+INFO is a plist for filling up Org-roam's capture templates.
+NODE is an `org-roam-node' construct containing information about the node.
+PROPS is a plist containing additional Org-roam properties for each template.
+TEMPLATES is a list of org-roam templates."
+  (let* ((props (plist-put props :call-location (point-marker)))
+         (org-capture-templates
+          (mapcar (lambda (template)
+                    (org-roam-capture--convert-template template props))
+                  (or templates org-roam-capture-templates)))
+         (org-roam-capture--node node)
+         (org-roam-capture--info info))
+    (when (and (not keys)
+               (= (length org-capture-templates) 1))
+      (setq keys (caar org-capture-templates)))
+    (org-capture goto keys)))
+
+;;;###autoload
+(cl-defun org-roam-capture (&optional goto keys &key filter-fn templates info)
+  "Launches an `org-capture' process for a new or existing node.
+This uses the templates defined at `org-roam-capture-templates'.
+Arguments GOTO and KEYS see `org-capture'.
+FILTER-FN is a function to filter out nodes: it takes an `org-roam-node',
+and when nil is returned the node will be filtered out.
+The TEMPLATES, if provided, override the list of capture templates (see
+`org-roam-capture-'.)
+The INFO, if provided, is passed along to the underlying `org-roam-capture-'."
+  (interactive "P")
+  (let ((node (org-roam-node-read nil filter-fn)))
+    (org-roam-capture- :goto goto
+                       :info info
+                       :keys keys
+                       :templates templates
+                       :node node
+                       :props '(:immediate-finish nil))))
+
+;;; Capture process
 (defun org-roam-capture-p ()
   "Return t if the current capture process is an Org-roam capture.
 This function is to only be called when `org-capture-plist' is
@@ -423,105 +441,53 @@ the capture)."
     (setq org-capture-plist
           (plist-put org-capture-plist :org-roam p))))
 
-;; FIXME: Pending upstream patch
+;;;; Capture target
-;; https://orgmode.org/list/87h7tv9pkm.fsf@hidden/T/#u
+(defun org-roam-capture--prepare-buffer ()
-;;
+  "Prepare the capture buffer for the current Org-roam based capture template.
-;; Org-capture's behaviour right now is that `org-capture-plist' is valid only
+This function will initialize and setup the capture buffer,
-;; during the initialization of the Org-capture buffer. The value of
+create the target node (`:if-new') if it doesn't exist, and place
-;; `org-capture-plist' is saved into buffer-local `org-capture-current-plist'.
+the point for further processing by `org-capture'.
-;; However, the value for that particular capture is no longer accessible for
-;; hooks in `org-capture-after-finalize-hook', since the capture buffer has been
-;; cleaned up.
-;;
-;; This advice restores the global `org-capture-plist' during finalization, so
-;; the plist is valid during both initialization and finalization of the
-;; capture.
-(defun org-roam-capture--update-plist (&optional _)
-  "Update global plist from local var."
-  (setq org-capture-plist org-capture-current-plist))
 
-(advice-add 'org-capture-finalize :before #'org-roam-capture--update-plist)
+Note: During the capture process this function is run by
+`org-capture-set-target-location', as a (function ...) based
+capture target."
+  (let ((id (cond ((run-hook-with-args-until-success 'org-roam-capture-preface-hook))
+                  ((and (org-roam-node-file org-roam-capture--node)
+                        (org-roam-node-point org-roam-capture--node))
+                   (set-buffer (org-capture-target-buffer (org-roam-node-file org-roam-capture--node)))
+                   (goto-char (org-roam-node-point org-roam-capture--node))
+                   (widen)
+                   (org-roam-node-id org-roam-capture--node))
+                  (t
+                   (org-roam-capture--setup-target-location)))))
+    (org-roam-capture--adjust-point-for-capture-type)
+    (org-capture-put :template
+                     (org-roam-capture--fill-template (org-capture-get :template)))
+    (org-roam-capture--put :id id)
+    (org-roam-capture--put :finalize (or (org-capture-get :finalize)
+                                         (org-roam-capture--get :finalize)))))
 
-(defun org-roam-capture--finalize-find-file ()
+(defun org-roam-capture--setup-target-location ()
-  "Visit the buffer after Org-capture is done.
-This function is to be called in the Org-capture finalization process.
-ID is unused."
-  (switch-to-buffer (org-capture-get :buffer)))
-
-(defun org-roam-capture--finalize-insert-link ()
-  "Insert a link to ID into the buffer where Org-capture was called.
-ID is the Org id of the newly captured content.
-This function is to be called in the Org-capture finalization process."
-  (when-let* ((mkr (org-roam-capture--get :call-location))
-              (buf (marker-buffer mkr)))
-    (with-current-buffer buf
-      (when-let ((region (org-roam-capture--get :region)))
-        (org-roam-unshield-region (car region) (cdr region))
-        (delete-region (car region) (cdr region))
-        (set-marker (car region) nil)
-        (set-marker (cdr region) nil))
-      (org-with-point-at mkr
-        (insert (org-link-make-string (concat "id:" (org-roam-capture--get :id))
-                                      (org-roam-capture--get :link-description)))))))
-
-(defun org-roam-capture--finalize ()
-  "Finalize the `org-roam-capture' process."
-  (when-let ((region (org-roam-capture--get :region)))
-    (org-roam-unshield-region (car region) (cdr region)))
-  (unless org-note-abort
-    (when-let ((finalize (org-roam-capture--get :finalize)))
-      (funcall (intern (concat "org-roam-capture--finalize-"
-                               (symbol-name (org-roam-capture--get :finalize)))))))
-  (remove-hook 'org-capture-after-finalize-hook #'org-roam-capture--finalize))
-
-(defun org-roam-capture--install-finalize ()
-  "Install `org-roam-capture--finalize' if the capture is an Org-roam capture."
-  (when (org-roam-capture-p)
-    (add-hook 'org-capture-after-finalize-hook #'org-roam-capture--finalize)))
-
-(add-hook 'org-capture-prepare-finalize-hook #'org-roam-capture--install-finalize)
-
-(defun org-roam-capture--fill-template (template &optional org-capture-p)
-  "Expand TEMPLATE and return it.
-It expands ${var} occurrences in TEMPLATE. When ORG-CAPTURE-P,
-also run Org-capture's template expansion."
-  (funcall (if org-capture-p #'org-capture-fill-template #'identity)
-           (org-roam-format
-            template
-            (lambda (key)
-              (let ((fn (intern (concat "org-roam-node-" key)))
-                    (ksym (intern (concat ":" key))))
-                (cond
-                 ((fboundp fn)
-                  (funcall fn org-roam-capture--node))
-                 ((plist-get org-roam-capture--info ksym)
-                  (plist-get org-roam-capture--info ksym))
-                 (t (let ((r (completing-read (format "%s: " key) nil)))
-                      (plist-put org-roam-capture--info ksym r)
-                      r))))))))
-
-(defun org-roam-capture--insert-ref ()
-  "Insert the ref if any."
-  (when-let ((ref (plist-get org-roam-capture--info :ref)))
-    (org-roam-ref-add ref)))
-
-(defun org-roam-capture--goto-location ()
   "Initialize the buffer, and goto the location of the new capture.
 Return the ID of the location."
-  (let (p)
+  (let (p new-file-p)
     (pcase (or (org-roam-capture--get :if-new)
                (user-error "Template needs to specify `:if-new'"))
       (`(file ,path)
        (setq path (expand-file-name
                    (string-trim (org-roam-capture--fill-template path t))
                    org-roam-directory))
+       (setq new-file-p (org-roam-capture--new-file-p path))
+       (when new-file-p (org-roam-capture--put :new-file path))
        (set-buffer (org-capture-target-buffer path))
        (widen)
-       (setq p (point)))
+       (setq p (goto-char (point-min))))
       (`(file+olp ,path ,olp)
        (setq path (expand-file-name
                    (string-trim (org-roam-capture--fill-template path t))
                    org-roam-directory))
+       (setq new-file-p (org-roam-capture--new-file-p path))
+       (when new-file-p (org-roam-capture--put :new-file path))
        (set-buffer (org-capture-target-buffer path))
        (setq p (point-min))
        (let ((m (org-roam-capture-find-or-create-olp olp)))
@@ -531,37 +497,94 @@ Return the ID of the location."
        (setq path (expand-file-name
                    (string-trim (org-roam-capture--fill-template path t))
                    org-roam-directory))
-       (let ((exists-p (file-exists-p path)))
+       (setq new-file-p (org-roam-capture--new-file-p path))
-         (set-buffer (org-capture-target-buffer path))
+       (set-buffer (org-capture-target-buffer path))
-         (unless exists-p
+       (when new-file-p
-           (insert (org-roam-capture--fill-template head t))))
+         (org-roam-capture--put :new-file path)
+         (insert (org-roam-capture--fill-template head t)))
        (widen)
-       (setq p (point-min)))
+       (setq p (goto-char (point-min))))
       (`(file+head+olp ,path ,head ,olp)
        (setq path (expand-file-name
                    (string-trim (org-roam-capture--fill-template path t))
                    org-roam-directory))
+       (setq new-file-p (org-roam-capture--new-file-p path))
+       (set-buffer (org-capture-target-buffer path))
        (widen)
-       (let ((exists-p (file-exists-p path)))
+       (when new-file-p
-         (set-buffer (org-capture-target-buffer path))
+         (org-roam-capture--put :new-file path)
-         (unless exists-p
+         (insert (org-roam-capture--fill-template head t)))
-           (insert (org-roam-capture--fill-template head t))))
        (setq p (point-min))
        (let ((m (org-roam-capture-find-or-create-olp olp)))
          (goto-char m)))
+      (`(file+datetree ,path ,tree-type)
+       (setq path (expand-file-name
+                   (string-trim (org-roam-capture--fill-template path t))
+                   org-roam-directory))
+       (require 'org-datetree)
+       (widen)
+       (set-buffer (org-capture-target-buffer path))
+       (unless (file-exists-p path)
+         (org-roam-capture--put :new-file path))
+       (funcall
+        (pcase tree-type
+          (`week #'org-datetree-find-iso-week-create)
+          (`month #'org-datetree-find-month-create)
+          (_ #'org-datetree-find-date-create))
+        (calendar-gregorian-from-absolute
+         (cond
+          (org-overriding-default-time
+           ;; Use the overriding default time.
+           (time-to-days org-overriding-default-time))
+          ((org-capture-get :default-time)
+           (time-to-days (org-capture-get :default-time)))
+          ((org-capture-get :time-prompt)
+           ;; Prompt for date.  Bind `org-end-time-was-given' so
+           ;; that `org-read-date-analyze' handles the time range
+           ;; case and returns `prompt-time' with the start value.
+           (let* ((org-time-was-given nil)
+                  (org-end-time-was-given nil)
+                  (prompt-time (org-read-date
+                                nil t nil "Date for tree entry:")))
+             (org-capture-put
+              :default-time
+              (if (or org-time-was-given
+                      (= (time-to-days prompt-time) (org-today)))
+                  prompt-time
+                ;; Use 00:00 when no time is given for another
+                ;; date than today?
+                (apply #'encode-time 0 0
+                       org-extend-today-until
+                       (cl-cdddr (decode-time prompt-time)))))
+             (time-to-days prompt-time)))
+          (t
+           ;; Current date, possibly corrected for late night
+           ;; workers.
+           (org-today)))))
+       (setq p (point)))
       (`(node ,title-or-id)
        ;; first try to get ID, then try to get title/alias
        (let ((node (or (org-roam-node-from-id title-or-id)
                        (org-roam-node-from-title-or-alias title-or-id)
-                       (user-error "No node with title or id \"%s\" title-or-id"))))
+                       (user-error "No node with title or id \"%s\"" title-or-id))))
          (set-buffer (org-capture-target-buffer (org-roam-node-file node)))
          (goto-char (org-roam-node-point node))
-         (setq p (org-roam-node-point node))
+         (setq p (org-roam-node-point node)))))
-         (org-end-of-subtree t t))))
+    ;; Setup `org-id' for the current capture target and return it back to the
+    ;; caller.
     (save-excursion
       (goto-char p)
-      (run-hooks 'org-roam-capture-new-node-hook)
+      (when-let* ((node org-roam-capture--node)
-      (org-id-get-create))))
+                  (id (org-roam-node-id node)))
+        (org-entry-put p "ID" id))
+      (prog1
+          (org-id-get-create)
+        (run-hooks 'org-roam-capture-new-node-hook)))))
+
+(defun org-roam-capture--new-file-p (path)
+  "Return t if PATH is for a new file with no visiting buffer."
+  (not (or (file-exists-p path)
+           (org-find-base-buffer-visiting path))))
 
 (defun org-roam-capture-find-or-create-olp (olp)
   "Return a marker pointing to the entry at OLP in the current buffer.
@@ -592,8 +615,10 @@ you can catch it with `condition-case'."
            ;; Create heading if it doesn't exist
            (goto-char end)
            (unless (bolp) (newline))
-           (org-insert-heading nil nil t)
+           (let (org-insert-heading-respect-content)
-           (unless (= lmax 1) (org-do-demote))
+             (org-insert-heading nil nil t))
+           (unless (= lmax 1)
+             (dotimes (_ level) (org-do-demote)))
            (insert heading)
            (setq end (point))
            (goto-char start)
@@ -605,61 +630,119 @@ you can catch it with `condition-case'."
        (setq lmin (1+ flevel) lmax (+ lmin (if org-odd-levels-only 1 0)))
        (setq start found
              end (save-excursion (org-end-of-subtree t t))))
-     (copy-marker end))))
+     (point-marker))))
 
-(defun org-roam-capture--get-node-from-ref (ref)
+(defun org-roam-capture--adjust-point-for-capture-type (&optional pos)
-  "Return the node from reference REF."
+  "Reposition the point for template insertion dependently on the capture type.
-  (save-match-data
+Return the newly adjusted position of `point'.
-    (when (string-match org-link-plain-re ref)
-      (let ((type (match-string 1 ref))
-            (path (match-string 2 ref)))
-        (when-let ((id (caar (org-roam-db-query
-                              [:select [nodes:id]
-                               :from refs
-                               :left-join nodes
-                               :on (= refs:node-id nodes:id)
-                               :where (= refs:type $s1)
-                               :and (= refs:ref $s2)
-                               :limit 1]
-                              type path))))
-          (org-roam-populate (org-roam-node-create :id id)))))))
 
-(defun org-roam-capture--get-point ()
+POS is the current position of point (an integer) inside the
-  "Return exact point to file for org-capture-template.
+currently active capture buffer, where the adjustment should
-This function is used solely in Org-roam's capture templates: see
+start to begin from. If it's nil, then it will default to
-`org-roam-capture-templates'."
+the current value of `point'."
-  (let ((id (cond ((plist-get org-roam-capture--info :ref)
+  (or pos (setq pos (point)))
-                   (if-let ((node (org-roam-capture--get-node-from-ref
+  (goto-char pos)
-                                   (plist-get org-roam-capture--info :ref))))
+  (let ((location-type (if (= pos 1) 'beginning-of-file 'heading-at-point)))
-                       (progn
+    (and (eq location-type 'heading-at-point)
-                         (set-buffer (org-capture-target-buffer (org-roam-node-file node)))
+         (cl-assert (org-at-heading-p)))
-                         (goto-char (org-roam-node-point node))
+    (pcase (org-capture-get :type)
-                         (widen)
+      (`plain
-                         (org-end-of-subtree t t))
+       (cl-case location-type
-                     (org-roam-capture--goto-location)))
+         (beginning-of-file
-                  ((and (org-roam-node-file org-roam-capture--node)
+          (if (org-capture-get :prepend)
-                        (org-roam-node-point org-roam-capture--node))
+              (let ((el (org-element-at-point)))
-                   (set-buffer (org-capture-target-buffer (org-roam-node-file org-roam-capture--node)))
+                (while (and (not (eobp))
-                   (goto-char (org-roam-node-point org-roam-capture--node))
+                            (memq (org-element-type el)
-                   (widen)
+                                  '(drawer property-drawer keyword comment comment-block horizontal-rule)))
-                   (org-end-of-subtree t t)
+                  (goto-char (org-element-property :end el))
-                   (org-roam-node-id org-roam-capture--node))
+                  (setq el (org-element-at-point))))
-                  (t
+            (goto-char (org-entry-end-position))))
-                   (org-roam-capture--goto-location)))))
+         (heading-at-point
-    (org-capture-put :template
+          (if (org-capture-get :prepend)
-                     (org-roam-capture--fill-template (org-capture-get :template)))
+              (org-end-of-meta-data t)
-    (org-roam-capture--put :id id)
+            (goto-char (org-entry-end-position))))))))
-    (org-roam-capture--put :finalize (or (org-capture-get :finalize)
+  (point))
-                                         (org-roam-capture--get :finalize)))))
+
+;;;; Finalizers
+(add-hook 'org-capture-prepare-finalize-hook #'org-roam-capture--install-finalize-h)
+(defun org-roam-capture--install-finalize-h ()
+  "Install `org-roam-capture--finalize' if the capture is an Org-roam capture."
+  (when (org-roam-capture-p)
+    (add-hook 'org-capture-after-finalize-hook #'org-roam-capture--finalize)))
+
+(defun org-roam-capture--finalize ()
+  "Finalize the `org-roam-capture' process."
+  (when-let ((region (org-roam-capture--get :region)))
+    (org-roam-unshield-region (car region) (cdr region)))
+  (if org-note-abort
+      (when-let ((new-file (org-roam-capture--get :new-file)))
+        (org-roam-message "Deleting file for aborted capture %s" new-file)
+        (when (find-buffer-visiting new-file)
+          (kill-buffer (find-buffer-visiting new-file)))
+        (delete-file new-file))
+    (when-let* ((finalize (org-roam-capture--get :finalize))
+                (org-roam-finalize-fn (intern (concat "org-roam-capture--finalize-"
+                                                      (symbol-name finalize)))))
+      (if (functionp org-roam-finalize-fn)
+          (funcall org-roam-finalize-fn)
+        (funcall finalize))))
+  (remove-hook 'org-capture-after-finalize-hook #'org-roam-capture--finalize))
+
+(defun org-roam-capture--finalize-find-file ()
+  "Visit the buffer after Org-capture is done.
+This function is to be called in the Org-capture finalization process.
+ID is unused."
+  (switch-to-buffer (org-capture-get :buffer)))
+
+(defun org-roam-capture--finalize-insert-link ()
+  "Insert a link to ID into the buffer where Org-capture was called.
+ID is the Org id of the newly captured content.
+This function is to be called in the Org-capture finalization process."
+  (when-let* ((mkr (org-roam-capture--get :call-location))
+              (buf (marker-buffer mkr)))
+    (with-current-buffer buf
+      (when-let ((region (org-roam-capture--get :region)))
+        (org-roam-unshield-region (car region) (cdr region))
+        (delete-region (car region) (cdr region))
+        (set-marker (car region) nil)
+        (set-marker (cdr region) nil))
+      (org-with-point-at mkr
+        (insert (org-link-make-string (concat "id:" (org-roam-capture--get :id))
+                                      (org-roam-capture--get :link-description)))))))
+
+;;;; Processing of the capture templates
+(defun org-roam-capture--fill-template (template &optional org-capture-p)
+  "Expand TEMPLATE and return it.
+It expands ${var} occurrences in TEMPLATE. When ORG-CAPTURE-P,
+also run Org-capture's template expansion."
+  (funcall (if org-capture-p #'org-capture-fill-template #'identity)
+           (org-roam-format-template
+            template
+            (lambda (key default-val)
+              (let ((fn (intern key))
+                    (node-fn (intern (concat "org-roam-node-" key)))
+                    (ksym (intern (concat ":" key))))
+                (cond
+                 ((fboundp fn)
+                  (funcall fn org-roam-capture--node))
+                 ((fboundp node-fn)
+                  (funcall node-fn org-roam-capture--node))
+                 ((plist-get org-roam-capture--info ksym)
+                  (plist-get org-roam-capture--info ksym))
+                 (t (let ((r (completing-read (format "%s: " key) nil nil nil default-val)))
+                      (plist-put org-roam-capture--info ksym r)
+                      r))))))))
 
 (defun org-roam-capture--convert-template (template &optional props)
   "Convert TEMPLATE from Org-roam syntax to `org-capture-templates' syntax.
 PROPS is a plist containing additional Org-roam specific
 properties to be added to the template."
   (pcase template
-    (`(,key ,desc)
+    (`(,_key ,_desc)
      template)
-    (`(,key ,desc ,type ,body . ,rest)
+    ((or `(,key ,desc ,type ignore ,body . ,rest)
+         `(,key ,desc ,type (function ignore) ,body . ,rest)
+         `(,key ,desc ,type ,body . ,rest))
      (setq rest (append rest props))
      (let (org-roam-plist options)
        (while rest
@@ -672,43 +755,12 @@ properties to be added to the template."
            (if custom
                (setq org-roam-plist (plist-put org-roam-plist key val))
              (setq options (plist-put options key val)))))
-       (append `(,key ,desc ,type #'org-roam-capture--get-point ,body)
+       (append `(,key ,desc ,type #'org-roam-capture--prepare-buffer ,body)
                options
                (list :org-roam org-roam-plist))))
     (_
      (signal 'invalid-template template))))
 
-;;;###autoload
-(cl-defun org-roam-capture- (&key goto keys node info props templates)
-  "Main entry point.
-GOTO and KEYS correspond to `org-capture' arguments.
-INFO is an alist for filling up Org-roam's capture templates.
-NODE is an `org-roam-node' construct containing information about the node.
-PROPS is a plist containing additional Org-roam properties for each template.
-TEMPLATES is a list of org-roam templates."
-  (let* ((props (plist-put props :call-location (point-marker)))
-         (org-capture-templates
-          (mapcar (lambda (template)
-                    (org-roam-capture--convert-template template props))
-                  (or templates org-roam-capture-templates)))
-         (org-roam-capture--node node)
-         (org-roam-capture--info info))
-    (when (and (not keys)
-               (= (length org-capture-templates) 1))
-      (setq keys (caar org-capture-templates)))
-    (org-capture goto keys)))
-
-;;;###autoload
-(defun org-roam-capture (&optional goto keys)
-  "Launches an `org-capture' process for a new or existing note.
-This uses the templates defined at `org-roam-capture-templates'.
-Arguments GOTO and KEYS see `org-capture'."
-  (interactive "P")
-  (let ((node (org-roam-node-read)))
-    (org-roam-capture- :goto goto
-                       :keys keys
-                       :node node
-                       :props '(:immediate-finish nil))))
 
 (provide 'org-roam-capture)
 

org-roam-compat.el

@@ -1,12 +1,12 @@
-;;; org-roam-compat.el --- Compatibility Code -*- coding: utf-8; lexical-binding: t; -*-
+;;; org-roam-compat.el --- Backward compatibility code -*- coding: utf-8; lexical-binding: t; -*-
 
-;; Copyright © 2020 Jethro Kuan <jethrokuan95@gmail.com>
+;; Copyright © 2020-2021 Jethro Kuan <jethrokuan95@gmail.com>
 
 ;; Author: Jethro Kuan <jethrokuan95@gmail.com>
 ;; URL: https://github.com/org-roam/org-roam
 ;; Keywords: org-mode, roam, convenience
-;; Version: 2.0.0
+;; Version: 2.1.0
-;; Package-Requires: ((emacs "26.1") (dash "2.13") (f "0.17.2") (org "9.4") (emacsql "3.0.0") (emacsql-sqlite3 "1.0.2") (magit-section "2.90.1"))
+;; Package-Requires: ((emacs "26.1"))
 
 ;; This file is NOT part of GNU Emacs.
 
@@ -27,14 +27,180 @@
 
 ;;; Commentary:
 ;;
-;; This file contains code needed for backward compatibility with older Emacsen
+;; This file is dedicated to maintain backward compatibility with older older
-;; and previous versions of org-roam.
+;; Emacsen and Org-roam versions.
 ;;
 ;;; Code:
-;;;; Library Requires
+;;; Backports
+;; REVIEW Remove when 26.x support is dropped. This is exact the same as
+;; `directory-files-recursively' from Emacs 26, but with FOLLOW-SYMLINKS
+;; parameter from Emacs 27.
+(defun org-roam--directory-files-recursively (dir regexp
+                                                  &optional include-directories predicate
+                                                  follow-symlinks)
+  "Return list of all files under directory DIR whose names match REGEXP.
+This function works recursively.  Files are returned in \"depth
+first\" order, and files from each directory are sorted in
+alphabetical order.  Each file name appears in the returned list
+in its absolute form.
+
+By default, the returned list excludes directories, but if
+optional argument INCLUDE-DIRECTORIES is non-nil, they are
+included.
+
+PREDICATE can be either nil (which means that all subdirectories
+of DIR are descended into), t (which means that subdirectories that
+can't be read are ignored), or a function (which is called with
+the name of each subdirectory, and should return non-nil if the
+subdirectory is to be descended into).
+
+If FOLLOW-SYMLINKS is non-nil, symbolic links that point to
+directories are followed.  Note that this can lead to infinite
+recursion."
+  (let* ((result nil)
+         (files nil)
+         (dir (directory-file-name dir))
+         ;; When DIR is "/", remote file names like "/method:" could
+         ;; also be offered.  We shall suppress them.
+         (tramp-mode (and tramp-mode (file-remote-p (expand-file-name dir)))))
+    (dolist (file (sort (file-name-all-completions "" dir)
+                        'string<))
+      (unless (member file '("./" "../"))
+        (if (directory-name-p file)
+            (let* ((leaf (substring file 0 (1- (length file))))
+                   (full-file (concat dir "/" leaf)))
+              ;; Don't follow symlinks to other directories.
+              (when (and (or (not (file-symlink-p full-file))
+                             (and (file-symlink-p full-file)
+                                  follow-symlinks))
+                         ;; Allow filtering subdirectories.
+                         (or (eq predicate nil)
+                             (eq predicate t)
+                             (funcall predicate full-file)))
+                (let ((sub-files
+                       (if (eq predicate t)
+                           (condition-case nil
+                               (org-roam--directory-files-recursively
+                                full-file regexp include-directories
+                                predicate follow-symlinks)
+                             (file-error nil))
+                         (org-roam--directory-files-recursively
+                          full-file regexp include-directories
+                          predicate follow-symlinks))))
+                  (setq result (nconc result sub-files))))
+              (when (and include-directories
+                         (string-match regexp leaf))
+                (setq result (nconc result (list full-file)))))
+          (when (string-match regexp file)
+            (push (concat dir "/" file) files)))))
+    (nconc result (nreverse files))))
+
+;;; Compatibility hacks and patches
+(advice-add #'org-id-add-location :around #'org-roam--handle-absent-org-id-locations-file-a)
+(defun org-roam--handle-absent-org-id-locations-file-a (fn &rest args)
+  "Gracefully handle errors related to absence of `org-id-locations-file'.
+FN is `org-id-locations-file' that comes from advice and ARGS are
+passed to it."
+  (let (result)
+    ;; Use `unwind-protect' over `condition-case' because `org-id' can produce various other errors, but all
+    ;; of its errors are generic ones, so trapping all of them isn't a good idea and preserving the correct
+    ;; backtrace is valuable.
+    (unwind-protect (setq result (apply fn args))
+      (unless result
+        (unless org-id-locations
+          ;; Pre-allocate the hash table to avoid weird access related errors during the regeneration.
+          (setq org-id-locations (make-hash-table :type 'equal)))
+        ;; `org-id' makes the assumption that `org-id-locations-file' will be stored in `user-emacs-directory'
+        ;; which always exist if you have Emacs, so it uses `with-temp-file' to write to the file. However,
+        ;; the users *do* change the path to this file and `with-temp-file' unable to create the file, if the
+        ;; path to it consists of directories that don't exist. We'll have to handle this ourselves.
+        (unless (file-exists-p (file-truename org-id-locations-file))
+          ;; If permissions allow that, try to create the user specified directory path to
+          ;; `org-id-locations-file' ourselves.
+          (condition-case _err
+              (progn (org-roam-message (concat "`org-id-locations-file' (%s) doesn't exist. "
+                                               "Trying to regenerate it (this may take a while)...")
+                                       org-id-locations-file)
+                     (make-directory (file-name-directory (file-truename org-id-locations-file)))
+                     (org-roam-update-org-id-locations)
+                     (apply fn args))
+            ;; In case of failure (lack of permissions), we'll patch it to at least handle the current session
+            ;; without errors.
+            (file-error (org-roam-message "Failed to regenerate `org-id-locations-file'")
+                        (lwarn 'org-roam :error "
+--------
+WARNING: `org-id-locations-file' (%s) doesn't exist!
+         Org-roam is unable to create it for you.
+--------
+
+This happens when Emacs doesn't have permissions to create the
+path to your `org-id-locations-file'. Org-roam will now fallback
+storing the file in your current `org-roam-directory', but the
+warning will keep popup with each new session.
+
+To stop this warning from popping up, set `org-id-locations-file'
+to the location you want and ensure that the path exists on your
+filesystem, then run M-x `org-roam-update-org-id-locations'.
+
+Note: While Org-roam doesn't depend on `org-id-locations-file' to
+lookup IDs for the nodes that are stored in the database, it
+still tries to keep it updated so IDs work across other files in
+Org-mode, so the IDs used in your `org-roam-directory' would be
+able to cross-reference outside of `org-roam-directory'. It also
+allows to keep linking with \"id:\" links within the current
+`org-roam-directory' to headings and files that are excluded from
+identification (e.g. with \"ROAM_EXCLUDE\" property) as Org-roam
+nodes." org-id-locations-file)
+                        (setq org-id-locations-file
+                              (expand-file-name ".orgids" (file-truename org-roam-directory)))
+                        (apply fn args)))))
+      result)))
 
 ;;; Obsolete aliases (remove after next major release)
+(define-obsolete-function-alias
+  'org-roam-setup
+  'org-roam-db-autosync-enable "org-roam 2.0")
+(define-obsolete-function-alias
+  'org-roam-teardown
+  'org-roam-db-autosync-disable "org-roam 2.0")
+
+(define-obsolete-variable-alias
+  'org-roam-current-node
+  'org-roam-buffer-current-node "org-roam 2.0")
+(define-obsolete-variable-alias
+  'org-roam-current-directory
+  'org-roam-buffer-current-directory "org-roam 2.0")
+(define-obsolete-function-alias
+  'org-roam-buffer-render
+  'org-roam-buffer-render-contents "org-roam 2.0")
+(define-obsolete-function-alias
+  'org-roam-buffer
+  'org-roam-buffer-display-dedicated "org-roam 2.0")
+(define-obsolete-function-alias
+  'org-roam-visit-thing
+  'org-roam-buffer-visit-thing "org-roam 2.0")
+
+(define-obsolete-function-alias
+  'org-roam-dailies-find-today
+  'org-roam-dailies-goto-today "org-roam 2.0")
+(define-obsolete-function-alias
+  'org-roam-dailies-find-yesterday
+  'org-roam-dailies-goto-yesterday "org-roam 2.0")
+(define-obsolete-function-alias
+  'org-roam-dailies-find-tomorrow
+  'org-roam-dailies-goto-tomorrow "org-roam 2.0")
+(define-obsolete-function-alias
+  'org-roam-dailies-find-next-note
+  'org-roam-dailies-goto-next-note "org-roam 2.0")
+(define-obsolete-function-alias
+  'org-roam-dailies-find-previous-note
+  'org-roam-dailies-goto-previous-note "org-roam 2.0")
+(define-obsolete-function-alias
+  'org-roam-dailies-find-date
+  'org-roam-dailies-goto-date "org-roam 2.0")
+
 ;;; Obsolete functions
+(make-obsolete 'org-roam-get-keyword 'org-collect-keywords "org-roam 2.0")
 
 (provide 'org-roam-compat)
 

org-roam-completion.el

@@ -1,133 +0,0 @@
-;;; org-roam-completion.el --- Completion features -*- coding: utf-8; lexical-binding: t; -*-
-
-;; Copyright © 2020 Jethro Kuan <jethrokuan95@gmail.com>
-
-;; Author: Jethro Kuan <jethrokuan95@gmail.com>
-;; URL: https://github.com/org-roam/org-roam
-;; Keywords: org-mode, roam, convenience
-;; Version: 2.0.0
-;; Package-Requires: ((emacs "26.1") (dash "2.13") (f "0.17.2") (org "9.4") (emacsql "3.0.0") (emacsql-sqlite3 "1.0.2") (magit-section "2.90.1"))
-;; This file is NOT part of GNU Emacs.
-
-;; This program is free software; you can redistribute it and/or modify
-;; it under the terms of the GNU General Public License as published by
-;; the Free Software Foundation; either version 3, or (at your option)
-;; any later version.
-;;
-;; This program is distributed in the hope that it will be useful,
-;; but WITHOUT ANY WARRANTY; without even the implied warranty of
-;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-;; GNU General Public License for more details.
-;;
-;; You should have received a copy of the GNU General Public License
-;; along with GNU Emacs; see the file COPYING.  If not, write to the
-;; Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor,
-;; Boston, MA 02110-1301, USA.
-
-;;; Commentary:
-;;
-;; This library provides completion-at-point functions for Org-roam.
-;;
-;; The two main functions provided to capf are:
-;;
-;;    `org-roam-complete-link-at-point' provides completions to nodes
-;;    within link brackets
-;;
-;;    `org-roam-complete-everywhere' provides completions for nodes everywhere,
-;;    matching the symbol at point
-;;
-;;; Code:
-(require 'cl-lib)
-(require 'org-element)
-
-(declare-function org-roam--get-titles "org-roam")
-
-(defcustom org-roam-completion-ignore-case t
-  "Whether to ignore case in Org-roam `completion-at-point' completions."
-  :group 'org-roam
-  :type 'boolean)
-
-(defcustom org-roam-completion-everywhere nil
-  "When non-nil, provide link completion matching outside of Org links."
-  :group 'org-roam
-  :type 'boolean)
-
-(defvar org-roam-completion-functions (list #'org-roam-complete-link-at-point
-                                            #'org-roam-complete-everywhere)
-  "List of functions to be used with `completion-at-point' for Org-roam.")
-
-(defun org-roam-complete-everywhere ()
-  "Provides completions for links for any word at point.
-This is a `completion-at-point' function, and is active when
-`org-roam-completion-everywhere' is non-nil."
-  (let ((end (point))
-        (start (point))
-        (exit-fn (lambda (&rest _) nil))
-        collection)
-    (when (and org-roam-completion-everywhere
-               (thing-at-point 'word)
-               (not (save-match-data (org-in-regexp org-link-any-re))))
-      (let ((bounds (bounds-of-thing-at-point 'word)))
-        (setq start (car bounds)
-              end (cdr bounds)
-              collection #'org-roam--get-titles
-              exit-fn (lambda (str _status)
-                        (delete-char (- (length str)))
-                        (insert "[[roam:" str "]]")))))
-    (when collection
-      (let ((prefix (buffer-substring-no-properties start end)))
-        (list start end
-              (if (functionp collection)
-                  (completion-table-case-fold
-                   (completion-table-dynamic
-                    (lambda (_)
-                      (cl-remove-if (apply-partially #'string= prefix)
-                                    (funcall collection))))
-                   (not org-roam-completion-ignore-case))
-                collection)
-              :exit-function exit-fn)))))
-
-(defun org-roam-complete-link-at-point ()
-  "Do appropriate completion for the link at point."
-  (let ((end (point))
-        (start (point))
-        collection link-type)
-    (when (org-in-regexp org-link-bracket-re 1)
-      (setq start (match-beginning 1)
-            end (match-end 1))
-      (let ((context (org-element-context)))
-        (pcase (org-element-lineage context '(link) t)
-          (`nil nil)
-          (link
-           (setq link-type (org-element-property :type link))
-           (when (member link-type '("roam" "fuzzy"))
-             (when (string= link-type "roam") (setq start (+ start (length "roam:"))))
-             (setq collection #'org-roam--get-titles))))))
-    (when collection
-      (let ((prefix (buffer-substring-no-properties start end)))
-        (list start end
-              (if (functionp collection)
-                  (completion-table-case-fold
-                   (completion-table-dynamic
-                    (lambda (_)
-                      (cl-remove-if (apply-partially #'string= prefix)
-                                    (funcall collection))))
-                   (not org-roam-completion-ignore-case))
-                collection)
-              :exit-function
-              (lambda (str &rest _)
-                (delete-char (- 0 (length str)))
-                (insert (concat (unless (string= link-type "roam") "roam:")
-                                str))
-                (forward-char 2)))))))
-
-(defun org-roam--register-completion-functions ()
-  "."
-  (dolist (fn org-roam-completion-functions)
-    (add-hook 'completion-at-point-functions fn nil t)))
-
-(add-hook 'org-roam-find-file-hook #'org-roam--register-completion-functions)
-
-(provide 'org-roam-completion)
-
-;;; org-roam-completion.el ends here

org-roam-db.el

@@ -1,12 +1,12 @@
 ;;; org-roam-db.el --- Org-roam database API -*- coding: utf-8; lexical-binding: t; -*-
 
-;; Copyright © 2020 Jethro Kuan <jethrokuan95@gmail.com>
+;; Copyright © 2020-2021 Jethro Kuan <jethrokuan95@gmail.com>
 
 ;; Author: Jethro Kuan <jethrokuan95@gmail.com>
 ;; URL: https://github.com/org-roam/org-roam
 ;; Keywords: org-mode, roam, convenience
-;; Version: 2.0.0
+;; Version: 2.1.0
-;; Package-Requires: ((emacs "26.1") (dash "2.13") (f "0.17.2") (org "9.4") (emacsql "3.0.0") (emacsql-sqlite3 "1.0.2") (magit-section "2.90.1"))
+;; Package-Requires: ((emacs "26.1") (dash "2.13") (f "0.17.2") (org "9.4") (emacsql "3.0.0") (emacsql-sqlite "1.0.0") (magit-section "2.90.1"))
 
 ;; This file is NOT part of GNU Emacs.
 
@@ -27,37 +27,15 @@
 
 ;;; Commentary:
 ;;
-;; This library provides the underlying database api to org-roam.
+;; This module provides the underlying database API to Org-roam.
 ;;
 ;;; Code:
-;;;; Library Requires
+(require 'org-roam)
-(eval-when-compile (require 'subr-x))
+(defvar org-outline-path-cache)
-(require 'emacsql)
-(require 'emacsql-sqlite3)
-(require 'seq)
 
-(eval-and-compile
+;;; Options
-  (require 'org-roam-macs)
-  ;; For `org-with-wide-buffer'
-  (require 'org-macs))
-(require 'org)
-(require 'ol)
-(require 'org-roam-utils)
-
-(defvar org-roam-find-file-hook)
-(defvar org-roam-directory)
-(defvar org-roam-verbose)
-(defvar org-agenda-files)
-
-(declare-function org-roam-id-at-point "org-roam")
-(declare-function org-roam--org-roam-file-p "org-roam")
-(declare-function org-roam--list-all-files "org-roam")
-(declare-function org-roam-node-at-point "org-roam")
-
-;;;; Options
 (defcustom org-roam-db-location (expand-file-name "org-roam.db" user-emacs-directory)
-  "The full path to file where the Org-roam database is stored.
+  "The path to file where the Org-roam database is stored.
-If this is non-nil, the Org-roam sqlite database is saved here.
 
 It is the user's responsibility to set this correctly, especially
 when used with multiple Org-roam instances."
@@ -66,26 +44,53 @@ when used with multiple Org-roam instances."
 
 (defcustom org-roam-db-gc-threshold gc-cons-threshold
   "The value to temporarily set the `gc-cons-threshold' threshold to.
-During large, heavy operations like `org-roam-db-sync',
+During `org-roam-db-sync', Emacs can pause multiple times to
-many GC operations happen because of the large number of
+perform garbage collection because of the large number of
-temporary structures generated (e.g. parsed ASTs). Temporarily
+temporary structures generated (e.g. parsed ASTs).
-increasing `gc-cons-threshold' will help reduce the number of GC
-operations, at the cost of temporary memory usage.
 
-This defaults to the original value of `gc-cons-threshold', but
+`gc-cons-threshold' is temporarily set to
-tweaking this number may lead to better overall performance. For
+`org-roam-db-gc-threshold' during this operation, and increasing
-example, to reduce the number of GCs, one may set it to a large
+`gc-cons-threshold' will help reduce the number of GC operations,
-value like `most-positive-fixnum'."
+at the cost of memory usage. Tweaking this value may lead to
+better overall performance.
+
+For example, to reduce the number of GCs to the minimum, on
+machines with large memory one may set it to
+`most-positive-fixnum'."
   :type 'int
   :group 'org-roam)
 
-(defconst org-roam-db--version 12)
+(defcustom org-roam-db-node-include-function (lambda () t)
+  "A custom function to check if the point contains a valid node.
+This function is called each time a node (both file and headline)
+is about to be saved into the Org-roam database.
+
+If the function returns nil, Org-roam will skip the node. This
+function is useful for excluding certain nodes from the Org-roam
+database."
+  :type 'function
+  :group 'org-roam)
+
+(defcustom org-roam-db-update-on-save t
+  "If t, update the Org-roam database upon saving the file.
+Disable this if your files are large and updating the database is
+slow."
+  :type 'boolean
+  :group 'org-roam)
+
+;;; Variables
+(defconst org-roam-db-version 16)
+
+;; TODO Rename this
+(defconst org-roam--sqlite-available-p
+  (with-demoted-errors "Org-roam initialization: %S"
+    (emacsql-sqlite-ensure-binary)
+    t))
 
 (defvar org-roam-db--connection (make-hash-table :test #'equal)
   "Database connection to Org-roam database.")
 
-;;;; Core Functions
+;;; Core Functions
-
 (defun org-roam-db--get-connection ()
   "Return the database connection, if any."
   (gethash (expand-file-name org-roam-directory)
@@ -99,7 +104,7 @@ Performs a database upgrade when required."
                (emacsql-live-p (org-roam-db--get-connection)))
     (let ((init-db (not (file-exists-p org-roam-db-location))))
       (make-directory (file-name-directory org-roam-db-location) t)
-      (let ((conn (emacsql-sqlite3 org-roam-db-location)))
+      (let ((conn (emacsql-sqlite org-roam-db-location)))
         (set-process-query-on-exit-flag (emacsql-process conn) nil)
         (puthash (expand-file-name org-roam-directory)
                  conn
@@ -109,71 +114,83 @@ Performs a database upgrade when required."
         (let* ((version (caar (emacsql conn "PRAGMA user_version")))
                (version (org-roam-db--upgrade-maybe conn version)))
           (cond
-           ((> version org-roam-db--version)
+           ((> version org-roam-db-version)
             (emacsql-close conn)
             (user-error
              "The Org-roam database was created with a newer Org-roam version.  "
              "You need to update the Org-roam package"))
-           ((< version org-roam-db--version)
+           ((< version org-roam-db-version)
             (emacsql-close conn)
             (error "BUG: The Org-roam database scheme changed %s"
                    "and there is no upgrade path")))))))
   (org-roam-db--get-connection))
 
-;;;; Entrypoint: (org-roam-db-query)
+;;; Entrypoint: (org-roam-db-query)
+(define-error 'emacsql-constraint "SQL constraint violation")
 (defun org-roam-db-query (sql &rest args)
   "Run SQL query on Org-roam database with ARGS.
 SQL can be either the emacsql vector representation, or a string."
-  (if  (stringp sql)
+  (apply #'emacsql (org-roam-db) sql args))
-      (emacsql (org-roam-db) (apply #'format sql args))
-    (apply #'emacsql (org-roam-db) sql args)))
 
-;;;; Schemata
+(defun org-roam-db-query! (handler sql &rest args)
-;; NOTE: Foreign key somehow doesn't work! Adding a file column to every table as a workaround.
+  "Run SQL query on Org-roam database with ARGS.
+SQL can be either the emacsql vector representation, or a string.
+The query is expected to be able to fail, in this situation, run HANDLER."
+  (condition-case err
+      (org-roam-db-query sql args)
+    (emacsql-constraint
+     (funcall handler err))))
+
+;;; Schemata
 (defconst org-roam-db--table-schemata
   '((files
      [(file :unique :primary-key)
-      (hash :not-null)])
+      (hash :not-null)
+      (atime :not-null)
+      (mtime :not-null)])
 
     (nodes
-     [(id :primary-key :not-null)
+     ([(id :not-null :primary-key)
-      (file :not-null)
+       (file :not-null)
-      (level :not-null)
+       (level :not-null)
-      (pos :not-null)
+       (pos :not-null)
-      todo
+       todo
-      priority
+       priority
-      (scheduled text)
+       (scheduled text)
-      (deadline text)
+       (deadline text)
-      title]
+       title
-     (:foreign-key [file] :references files [file] :on-delete :cascade))
+       properties
+       olp]
+      (:foreign-key [file] :references files [file] :on-delete :cascade)))
 
     (aliases
-     [(file :not-null)
+     ([(node-id :not-null)
-      (node-id :not-null)
+       alias]
-      alias]
+      (:foreign-key [node-id] :references nodes [id] :on-delete :cascade)))
-     (:foreign-key [node-id] :references nodes [id] :on-delete :cascade))
 
     (refs
-     ([(file :not-null)
+     ([(node-id :not-null)
-       (node-id :not-null)
        (ref :not-null)
        (type :not-null)]
       (:foreign-key [node-id] :references nodes [id] :on-delete :cascade)))
 
     (tags
-     [(file :not-null)
+     ([(node-id :not-null)
-      (node-id :not-null)
+       tag]
-      tag]
+      (:foreign-key [node-id] :references nodes [id] :on-delete :cascade)))
-     (:foreign-key [node-id] :references nodes [id] :on-delete :cascade))
 
     (links
-     [(file :not-null)
+     ([(pos :not-null)
-      (pos :not-null)
+       (source :not-null)
-      (source :not-null)
+       (dest :not-null)
-      (dest :not-null)
+       (type :not-null)
-      (type :not-null)
+       (properties :not-null)]
-      (properties :not-null)]
+      (:foreign-key [source] :references nodes [id] :on-delete :cascade)))))
-     (:foreign-key [file] :references files [file] :on-delete :cascade))))
+
+(defconst org-roam-db--table-indices
+  '((alias-node-id aliases [node-id])
+    (refs-node-id refs [node-id])
+    (tags-node-id tags [node-id])))
 
 (defun org-roam-db--init (db)
   "Initialize database DB with the correct schema and user version."
@@ -181,16 +198,18 @@ SQL can be either the emacsql vector representation, or a string."
     (emacsql db "PRAGMA foreign_keys = ON")
     (pcase-dolist (`(,table ,schema) org-roam-db--table-schemata)
       (emacsql db [:create-table $i1 $S2] table schema))
-    (emacsql db (format "PRAGMA user_version = %s" org-roam-db--version))))
+    (pcase-dolist (`(,index-name ,table ,columns) org-roam-db--table-indices)
+      (emacsql db [:create-index $i1 :on $i2 $S3] index-name table columns))
+    (emacsql db (format "PRAGMA user_version = %s" org-roam-db-version))))
 
 (defun org-roam-db--upgrade-maybe (db version)
   "Upgrades the database schema for DB, if VERSION is old."
   (emacsql-with-transaction db
     'ignore
-    (if (< version org-roam-db--version)
+    (if (< version org-roam-db-version)
         (progn
           (org-roam-message (format "Upgrading the Org-roam database from version %d to version %d"
-                                    version org-roam-db--version))
+                                    version org-roam-db-version))
           (org-roam-db-sync t))))
   version)
 
@@ -208,8 +227,8 @@ the current `org-roam-directory'."
   (dolist (conn (hash-table-values org-roam-db--connection))
     (org-roam-db--close conn)))
 
-;;;; Database API
+;;; Database API
-;;;;; Clearing
+;;;; Clearing
 (defun org-roam-db-clear-all ()
   "Clears all entries in the Org-roam cache."
   (interactive)
@@ -222,21 +241,23 @@ the current `org-roam-directory'."
 This is equivalent to removing the node from the graph.
 If FILE is nil, clear the current buffer."
   (setq file (or file (buffer-file-name (buffer-base-buffer))))
-  (dolist (table (mapcar #'car org-roam-db--table-schemata))
+  (org-roam-db-query [:delete :from files
-    (org-roam-db-query `[:delete :from ,table
+                      :where (= file $s1)]
-                         :where (= file $s1)]
+                     file))
-                       file)))
 
-;;;;; Updating tables
+;;;; Updating tables
 (defun org-roam-db-insert-file ()
   "Update the files table for the current buffer.
 If UPDATE-P is non-nil, first remove the file in the database."
   (let* ((file (buffer-file-name))
+         (attr (file-attributes file))
+         (atime (file-attribute-access-time attr))
+         (mtime (file-attribute-modification-time attr))
          (hash (org-roam-db--file-hash)))
     (org-roam-db-query
      [:insert :into files
       :values $v1]
-     (list (vector file hash)))))
+     (list (vector file hash atime mtime)))))
 
 (defun org-roam-db-get-scheduled-time ()
   "Return the scheduled time at point in ISO8601 format."
@@ -248,13 +269,20 @@ If UPDATE-P is non-nil, first remove the file in the database."
   (when-let ((time (org-get-deadline-time (point))))
     (org-format-time-string "%FT%T%z" time)))
 
-(defun org-roam-db-map-headlines (fns)
+(defun org-roam-db-node-p ()
-  "Run FNS over all headlines in the current buffer."
+  "Return t if headline at point is an Org-roam node, else return nil."
+  (and (org-id-get)
+       (not (cdr (assoc "ROAM_EXCLUDE" (org-entry-properties))))
+       (funcall org-roam-db-node-include-function)))
+
+(defun org-roam-db-map-nodes (fns)
+  "Run FNS over all nodes in the current buffer."
   (org-with-point-at 1
     (org-map-entries
      (lambda ()
-       (dolist (fn fns)
+       (when (org-roam-db-node-p)
-         (funcall fn))))))
+         (dolist (fn fns)
+           (funcall fn)))))))
 
 (defun org-roam-db-map-links (fns)
   "Run FNS over all links in the current buffer."
@@ -267,12 +295,14 @@ If UPDATE-P is non-nil, first remove the file in the database."
 (defun org-roam-db-insert-file-node ()
   "Insert the file-level node into the Org-roam cache."
   (org-with-point-at 1
-    (when (= (org-outline-level) 0)
+    (when (and (= (org-outline-level) 0)
+               (org-roam-db-node-p))
       (when-let ((id (org-id-get)))
         (let* ((file (buffer-file-name (buffer-base-buffer)))
-               (title (or (cadr (assoc "TITLE" (org-collect-keywords '("title"))
+               (title (org-link-display-format
-                                       #'string-equal))
+                       (or (cadr (assoc "TITLE" (org-collect-keywords '("title"))
-                          (file-relative-name file org-roam-directory)))
+                                        #'string-equal))
+                           (file-relative-name file org-roam-directory))))
                (pos (point))
                (todo nil)
                (priority nil)
@@ -281,48 +311,50 @@ If UPDATE-P is non-nil, first remove the file in the database."
                (level 0)
                (aliases (org-entry-get (point) "ROAM_ALIASES"))
                (tags org-file-tags)
-               (refs (org-entry-get (point) "ROAM_REFS")))
+               (refs (org-entry-get (point) "ROAM_REFS"))
-          (condition-case nil
+               (properties (org-entry-properties))
-              (progn
+               (olp nil))
+          (org-roam-db-query!
+           (lambda (err)
+             (lwarn 'org-roam :warning "%s for %s (%s) in %s"
+                    (error-message-string err)
+                    title id file))
+           [:insert :into nodes
+            :values $v1]
+           (vector id file level pos todo priority
+                   scheduled deadline title properties olp))
+          (when tags
+            (org-roam-db-query
+             [:insert :into tags
+              :values $v1]
+             (mapcar (lambda (tag)
+                       (vector id (substring-no-properties tag)))
+                     tags)))
+          (when aliases
+            (org-roam-db-query
+             [:insert :into aliases
+              :values $v1]
+             (mapcar (lambda (alias)
+                       (vector id alias))
+                     (split-string-and-unquote aliases))))
+          (when refs
+            (setq refs (split-string-and-unquote refs))
+            (let (rows)
+              (dolist (ref refs)
+                (if (string-match org-link-plain-re ref)
+                    (progn
+                      (push (vector id (match-string 2 ref)
+                                    (match-string 1 ref)) rows))
+                  (lwarn '(org-roam) :warning
+                         "%s:%s\tInvalid ref %s, skipping..."
+                         (buffer-file-name) (point) ref)))
+              (when rows
                 (org-roam-db-query
-                 [:insert :into nodes
+                 [:insert :into refs
                   :values $v1]
-                 (vector id file level pos todo priority
+                 rows)))))))))
-                         scheduled deadline title))
-                (when tags
-                  (org-roam-db-query
-                   [:insert :into tags
-                    :values $v1]
-                   (mapcar (lambda (tag)
-                             (vector file id (substring-no-properties tag)))
-                           tags)))
-                (when aliases
-                  (org-roam-db-query
-                   [:insert :into aliases
-                    :values $v1]
-                   (mapcar (lambda (alias)
-                             (vector file id alias))
-                           (split-string-and-unquote aliases))))
-                (when refs
-                  (setq refs (split-string-and-unquote refs))
-                  (let (rows)
-                    (dolist (ref refs)
-                      (if (string-match org-link-plain-re ref)
-                          (progn
-                            (push (vector file id (match-string 2 ref)
-                                          (match-string 1 ref)) rows))
-                        (lwarn '(org-roam) :warning
-                               "%s:%s\tInvalid ref %s, skipping..."
-                               (buffer-file-name) (point) ref)))
-                    (when rows
-                      (org-roam-db-query
-                       [:insert :into refs
-                        :values $v1]
-                       rows)))))
-            (t
-             (lwarn '(org-roam) :error "Duplicate ID %s, skipping..." id))))))))
 
-(defun org-roam-db-insert-node-data ()
+(cl-defun org-roam-db-insert-node-data ()
   "Insert node data for headline at point into the Org-roam cache."
   (when-let ((id (org-id-get)))
     (let* ((file (buffer-file-name (buffer-base-buffer)))
@@ -333,42 +365,47 @@ If UPDATE-P is non-nil, first remove the file in the database."
            (level (nth 1 heading-components))
            (scheduled (org-roam-db-get-scheduled-time))
            (deadline (org-roam-db-get-deadline-time))
-           (title (nth 4 heading-components)))
+           (title (or (nth 4 heading-components)
-      (condition-case nil
+                      (progn (lwarn 'org-roam :warning "Node in %s:%s:%s has no title, skipping..."
-          (org-roam-db-query
+                                    file
-           [:insert :into nodes
+                                    (line-number-at-pos)
-            :values $v1]
+                                    (1+ (- (point) (line-beginning-position))))
-           (vector id file level pos todo priority
+                             (cl-return-from org-roam-db-insert-node-data))))
-                   scheduled deadline title))
+           (properties (org-entry-properties))
-        (t
+           (olp (org-get-outline-path nil 'use-cache))
-         (lwarn '(org-roam) :error
+           (title (org-link-display-format title)))
-                "Duplicate ID %s, skipping..." id))))))
+      (org-roam-db-query!
+       (lambda (err)
+         (lwarn 'org-roam :warning "%s for %s (%s) in %s"
+                (error-message-string err)
+                title id file))
+       [:insert :into nodes
+        :values $v1]
+       (vector id file level pos todo priority
+               scheduled deadline title properties olp)))))
 
 (defun org-roam-db-insert-aliases ()
   "Insert aliases for node at point into Org-roam cache."
-  (when-let ((file (buffer-file-name (buffer-base-buffer)))
+  (when-let ((node-id (org-id-get))
-             (node-id (org-id-get))
              (aliases (org-entry-get (point) "ROAM_ALIASES")))
     (org-roam-db-query [:insert :into aliases
                         :values $v1]
                        (mapcar (lambda (alias)
-                                 (vector file node-id alias))
+                                 (vector node-id alias))
                                (split-string-and-unquote aliases)))))
 
 (defun org-roam-db-insert-tags ()
   "Insert tags for node at point into Org-roam cache."
-  (when-let ((file (buffer-file-name (buffer-base-buffer)))
+  (when-let ((node-id (org-id-get))
-             (node-id (org-id-get))
              (tags (org-get-tags)))
     (org-roam-db-query [:insert :into tags
                         :values $v1]
                        (mapcar (lambda (tag)
-                                 (vector file node-id tag)) tags))))
+                                 (vector node-id (substring-no-properties tag))) tags))))
 
 (defun org-roam-db-insert-refs ()
   "Insert refs for node at point into Org-roam cache."
-  (when-let* ((file (buffer-file-name (buffer-base-buffer)))
+  (when-let* ((node-id (org-id-get))
-              (node-id (org-id-get))
               (refs (org-entry-get (point) "ROAM_REFS"))
               (refs (split-string-and-unquote refs)))
     (let (rows)
@@ -376,7 +413,7 @@ If UPDATE-P is non-nil, first remove the file in the database."
         (save-match-data
           (if (string-match org-link-plain-re ref)
               (progn
-                (push (vector file node-id (match-string 2 ref) (match-string 1 ref)) rows))
+                (push (vector node-id (match-string 2 ref) (match-string 1 ref)) rows))
             (lwarn '(org-roam) :warning
                    "%s:%s\tInvalid ref %s, skipping..." (buffer-file-name) (point) ref))))
       (org-roam-db-query [:insert :into refs
@@ -387,18 +424,28 @@ If UPDATE-P is non-nil, first remove the file in the database."
   "Insert link data for LINK at current point into the Org-roam cache."
   (save-excursion
     (goto-char (org-element-property :begin link))
-    (let ((file (buffer-file-name (buffer-base-buffer)))
+    (let ((type (org-element-property :type link))
-          (type (org-element-property :type link))
+          (path (org-element-property :path link))
-          (dest (org-element-property :path link))
+          (properties (list :outline (ignore-errors
-          (properties (list :outline (org-get-outline-path)))
+                                       ;; This can error if link is not under any headline
+                                       (org-get-outline-path 'with-self 'use-cache))))
           (source (org-roam-id-at-point)))
-      (when source
+      ;; For Org-ref links, we need to split the path into the cite keys
+      (when (and (boundp 'org-ref-cite-types)
+                 (fboundp 'org-ref-split-and-strip-string)
+                 (member type org-ref-cite-types))
+        (setq path (org-ref-split-and-strip-string path)))
+      (unless (listp path)
+        (setq path (list path)))
+      (when (and source path)
         (org-roam-db-query
          [:insert :into links
           :values $v1]
-         (vector file (point) source dest type properties))))))
+         (mapcar (lambda (p)
+                   (vector (point) source p type properties))
+                 path))))))
 
-;;;;; Fetching
+;;;; Fetching
 (defun org-roam-db--get-current-files ()
   "Return a hash-table of file to the hash of its file contents."
   (let ((current-files (org-roam-db-query [:select [file hash] :from files]))
@@ -409,6 +456,10 @@ If UPDATE-P is non-nil, first remove the file in the database."
 
 (defun org-roam-db--file-hash (&optional file-path)
   "Compute the hash of FILE-PATH, a file or current buffer."
+  ;; If it is a GPG encrypted file, we always want to compute the hash
+  ;; for the GPG encrypted file (undecrypted)
+  (when (and (not file-path) (equal "gpg" (file-name-extension (buffer-file-name))))
+    (setq file-path (buffer-file-name)))
   (if file-path
       (with-temp-buffer
         (set-buffer-multibyte nil)
@@ -417,38 +468,7 @@ If UPDATE-P is non-nil, first remove the file in the database."
     (org-with-wide-buffer
      (secure-hash 'sha1 (current-buffer)))))
 
-;;;;; Updating
+;;;; Synchronization
-(defun org-roam-db-sync (&optional force)
-  "Synchronize the cache state with the current Org files on-disk.
-If FORCE, force a rebuild of the cache from scratch."
-  (interactive "P")
-  (when force (delete-file org-roam-db-location))
-  (org-roam-db--close) ;; Force a reconnect
-  (org-roam-db) ;; To initialize the database, no-op if already initialized
-  (let* ((gc-cons-threshold org-roam-db-gc-threshold)
-         (org-agenda-files nil)
-         (org-roam-files (org-roam--list-all-files))
-         (current-files (org-roam-db--get-current-files))
-         (modified-files nil))
-    (dolist (file org-roam-files)
-      (let ((contents-hash (org-roam-db--file-hash file)))
-        (unless (string= (gethash file current-files)
-                         contents-hash)
-          (push file modified-files)))
-      (remhash file current-files))
-    (if (fboundp 'dolist-with-progress-reporter)
-        (dolist-with-progress-reporter (file (hash-table-keys current-files))
-            "Clearing removed files..."
-          (org-roam-db-clear-file file))
-      (dolist (file (hash-table-keys current-files))
-        (org-roam-db-clear-file file)))
-    (if (fboundp 'dolist-with-progress-reporter)
-        (dolist-with-progress-reporter (file modified-files)
-            "Processing modified files..."
-          (org-roam-db-update-file file))
-      (dolist (file modified-files)
-        (org-roam-db-update-file file)))))
-
 (defun org-roam-db-update-file (&optional file-path)
   "Update Org-roam cache for FILE-PATH.
 If the file does not exist anymore, remove it from the cache.
@@ -464,21 +484,133 @@ If the file exists, update the cache with information."
           (org-roam-db-clear-file)
           (org-roam-db-insert-file)
           (org-roam-db-insert-file-node)
-          (org-roam-db-map-headlines
+          (setq org-outline-path-cache nil)
+          (org-roam-db-map-nodes
            (list #'org-roam-db-insert-node-data
                  #'org-roam-db-insert-aliases
                  #'org-roam-db-insert-tags
                  #'org-roam-db-insert-refs))
+          (setq org-outline-path-cache nil)
           (org-roam-db-map-links
            (list #'org-roam-db-insert-link)))))))
 
-(defun org-roam-db--update-on-save-h ()
+;;;###autoload
-  "."
+(defun org-roam-db-sync (&optional force)
-  (add-hook 'after-save-hook #'org-roam-db-update-file nil t))
+  "Synchronize the cache state with the current Org files on-disk.
+If FORCE, force a rebuild of the cache from scratch."
+  (interactive "P")
+  (org-roam-db--close) ;; Force a reconnect
+  (when force (delete-file org-roam-db-location))
+  (org-roam-db) ;; To initialize the database, no-op if already initialized
+  (let* ((gc-cons-threshold org-roam-db-gc-threshold)
+         (org-agenda-files nil)
+         (org-roam-files (org-roam-list-files))
+         (current-files (org-roam-db--get-current-files))
+         (modified-files nil))
+    (dolist (file org-roam-files)
+      (let ((contents-hash (org-roam-db--file-hash file)))
+        (unless (string= (gethash file current-files)
+                         contents-hash)
+          (push file modified-files)))
+      (remhash file current-files))
+    (emacsql-with-transaction (org-roam-db)
+      (if (fboundp 'dolist-with-progress-reporter)
+          (dolist-with-progress-reporter (file (hash-table-keys current-files))
+              "Clearing removed files..."
+            (org-roam-db-clear-file file))
+        (dolist (file (hash-table-keys current-files))
+          (org-roam-db-clear-file file)))
+      (if (fboundp 'dolist-with-progress-reporter)
+          (dolist-with-progress-reporter (file modified-files)
+              "Processing modified files..."
+            (org-roam-db-update-file file))
+        (dolist (file modified-files)
+          (org-roam-db-update-file file))))))
 
-(add-to-list 'org-roam-find-file-hook #'org-roam-db--update-on-save-h)
+;;;###autoload
+(define-minor-mode org-roam-db-autosync-mode
+  "Global minor mode to keep your Org-roam session automatically synchronized.
+Through the session this will continue to setup your
+buffers (that are Org-roam file visiting), keep track of the
+related changes, maintain cache consistency and incrementally
+update the currently active database.
 
-;; Diagnostic Interactives
+If you need to manually trigger resync of the currently active
+database, see `org-roam-db-sync' command."
+  :group 'org-roam
+  :global t
+  :init-value nil
+  (let ((enabled org-roam-db-autosync-mode))
+    (cond
+     (enabled
+      (add-hook 'find-file-hook  #'org-roam-db-autosync--setup-file-h)
+      (add-hook 'kill-emacs-hook #'org-roam-db--close-all)
+      (advice-add #'rename-file :after  #'org-roam-db-autosync--rename-file-a)
+      (advice-add #'delete-file :before #'org-roam-db-autosync--delete-file-a)
+      (org-roam-db-sync))
+     (t
+      (remove-hook 'find-file-hook  #'org-roam-db-autosync--setup-file-h)
+      (remove-hook 'kill-emacs-hook #'org-roam-db--close-all)
+      (advice-remove #'rename-file #'org-roam-db-autosync--rename-file-a)
+      (advice-remove #'delete-file #'org-roam-db-autosync--delete-file-a)
+      (org-roam-db--close-all)
+      ;; Disable local hooks for all org-roam buffers
+      (dolist (buf (org-roam-buffer-list))
+        (with-current-buffer buf
+          (remove-hook 'after-save-hook #'org-roam-db-autosync--try-update-on-save-h t)))))))
+
+;;;###autoload
+(defun org-roam-db-autosync-enable ()
+  "Activate `org-roam-db-autosync-mode'."
+  (org-roam-db-autosync-mode +1))
+
+(defun org-roam-db-autosync-disable ()
+  "Deactivate `org-roam-db-autosync-mode'."
+  (org-roam-db-autosync-mode -1))
+
+(defun org-roam-db-autosync-toggle ()
+  "Toggle `org-roam-db-autosync-mode' enabled/disabled."
+  (org-roam-db-autosync-mode 'toggle))
+
+(defun org-roam-db-autosync--delete-file-a (file &optional _trash)
+  "Maintain cache consistency when file deletes.
+FILE is removed from the database."
+  (when (and (not (auto-save-file-name-p file))
+             (not (backup-file-name-p file))
+             (org-roam-file-p file))
+    (org-roam-db-clear-file (expand-file-name file))))
+
+(defun org-roam-db-autosync--rename-file-a (old-file new-file-or-dir &rest _args)
+  "Maintain cache consistency of file rename.
+OLD-FILE is cleared from the database, and NEW-FILE-OR-DIR is added."
+  (let ((new-file (if (directory-name-p new-file-or-dir)
+                      (expand-file-name (file-name-nondirectory old-file) new-file-or-dir)
+                    new-file-or-dir)))
+    (setq new-file (expand-file-name new-file))
+    (setq old-file (expand-file-name old-file))
+    (when (and (not (auto-save-file-name-p old-file))
+               (not (auto-save-file-name-p new-file))
+               (not (backup-file-name-p old-file))
+               (not (backup-file-name-p new-file))
+               (org-roam-file-p old-file))
+      (org-roam-db-clear-file old-file))
+    (when (org-roam-file-p new-file)
+      (org-roam-db-update-file new-file))))
+
+(defun org-roam-db-autosync--setup-file-h ()
+  "Setup the current buffer if it visits an Org-roam file."
+  (when (org-roam-file-p) (run-hooks 'org-roam-find-file-hook)))
+
+(add-hook 'org-roam-find-file-hook #'org-roam-db-autosync--setup-update-on-save-h)
+(defun org-roam-db-autosync--setup-update-on-save-h ()
+  "Setup the current buffer to update the DB after saving the current file."
+  (add-hook 'after-save-hook #'org-roam-db-autosync--try-update-on-save-h nil t))
+
+(defun org-roam-db-autosync--try-update-on-save-h ()
+  "If appropriate, update the database for the current file after saving buffer."
+  (when org-roam-db-update-on-save (org-roam-db-update-file)))
+
+;;; Diagnostics
 (defun org-roam-db-diagnose-node ()
   "Print information about node at point."
   (interactive)

org-roam-doctor.el

@@ -1,221 +0,0 @@
-;;; org-roam-doctor.el --- Linter for Org-roam files -*- coding: utf-8; lexical-binding: t; -*-
-;;
-;; Copyright © 2020 Jethro Kuan <jethrokuan95@gmail.com>
-
-;; Author: Jethro Kuan <jethrokuan95@gmail.com>
-;; URL: https://github.com/jethrokuan/org-roam
-;; Keywords: org-mode, roam, convenience
-;; Version: 2.0.0
-;; Package-Requires: ((emacs "26.1") (dash "2.13") (f "0.17.2") (org "9.4") (emacsql "3.0.0") (emacsql-sqlite3 "1.0.2") (magit-section "2.90.1"))
-
-;; This file is NOT part of GNU Emacs.
-
-;; This program is free software; you can redistribute it and/or modify
-;; it under the terms of the GNU General Public License as published by
-;; the Free Software Foundation; either version 3, or (at your option)
-;; any later version.
-;;
-;; This program is distributed in the hope that it will be useful,
-;; but WITHOUT ANY WARRANTY; without even the implied warranty of
-;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-;; GNU General Public License for more details.
-;;
-;; You should have received a copy of the GNU General Public License
-;; along with GNU Emacs; see the file COPYING.  If not, write to the
-;; Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor,
-;; Boston, MA 02110-1301, USA.
-
-
-;;; Commentary:
-;;
-;; This library provides `org-roam-doctor', a utility for diagnosing and fixing
-;; Org-roam files. Running `org-roam-doctor' launches a list of checks defined
-;; by `org-roam-doctor--checkers'. Every checker is an instance of
-;; `org-roam-doctor-checker'.
-;;
-;; Each checker is given the Org parse tree (AST), and is expected to return a
-;; list of errors. The checker can also provide "actions" for auto-fixing errors
-;; (see `org-roam-doctor--remove-link' for an example).
-;;
-;; The UX experience is inspired by both org-lint and checkdoc, and their code
-;; is heavily referenced.
-;;
-;;; Code:
-;; Library Requires
-(require 'cl-lib)
-(require 'org)
-(require 'org-element)
-(require 'dash)
-(eval-when-compile
-  (require 'org-roam-macs))
-(require 'org-roam)
-
-(defvar org-roam-mode-map)
-
-(declare-function org-roam--get-roam-buffers "org-roam")
-(declare-function org-roam--list-all-files "org-roam")
-(declare-function org-roam--org-roam-file-p "org-roam")
-
-(defvar org-roam-verbose)
-
-(defcustom org-roam-doctor-inhibit-startup t
-  "Inhibit `org-mode' startup when processing files with `org-doctor'.
-When non-nil, images and LaTeX preview will not be generated,
-tables will not be aligned, and headlines will not respect
-startup visability. This significantly improves performance when
-processing multiple files"
-  :type 'boolean
-  :group 'org-roam)
-
-(cl-defstruct (org-roam-doctor-checker (:copier nil))
-  (name 'missing-checker-name)
-  (description "")
-  (actions nil))
-
-(defconst org-roam-doctor--checkers
-  (list
-   (make-org-roam-doctor-checker
-    :name 'org-roam-doctor-broken-links
-    :description "Fix broken links."
-    :actions '(("d" . ("Unlink" . org-roam-doctor--remove-link))
-               ("r" . ("Replace link" . org-roam-doctor--replace-link))))))
-
-(defun org-roam-doctor-broken-links (ast)
-  "Checker for detecting broken links.
-AST is the org-element parse tree."
-  (let (reports)
-    (org-element-map ast 'link
-      (lambda (l)
-        (when (equal "id" (org-element-property :type l))
-          (let ((id (org-element-property :path l)))
-            (unless (org-id-find id)
-              (push `(,(org-element-property :begin l)
-                      ,(format "Broken id link \"%s\"" id))
-                    reports))))))
-    reports))
-
-(defun org-roam-doctor--check (buffer checkers)
-  "Check BUFFER for errors.
-CHECKERS is the list of checkers used."
-  (with-current-buffer buffer
-    (save-excursion
-      (goto-char (point-min))
-      (let* ((ast (org-element-parse-buffer))
-             (errors (sort (cl-mapcan
-                            (lambda (c)
-                              (mapcar
-                               (lambda (report)
-                                 (list (set-marker (make-marker) (car report))
-                                       (nth 1 report) c))
-                               (save-excursion
-                                 (funcall
-                                  (org-roam-doctor-checker-name c)
-                                  ast))))
-                            checkers)
-                           #'car-less-than-car)))
-        (dolist (e errors)
-          (pcase-let ((`(,m ,msg ,checker) e))
-            (switch-to-buffer buffer)
-            (goto-char m)
-            (org-reveal)
-            (undo-boundary)
-            (org-roam-doctor--resolve msg checker)
-            (set-marker m nil)))
-        errors))))
-
-;;; Actions
-(defun org-roam-doctor--recursive-edit ()
-  "Launch into a recursive edit."
-  (message "When you're done editing press C-M-c to continue.")
-  (recursive-edit))
-
-(defun org-roam-doctor--skip ()
-  "Skip the current error."
-  (org-roam-message "Skipping..."))
-
-(defun org-roam-doctor--replace-link ()
-  "Replace the current link with a new link."
-  (save-match-data
-    (unless (org-in-regexp org-link-bracket-re 1)
-      (user-error "No link at point"))
-    (let ((orig (buffer-string))
-          (p (point)))
-      (condition-case nil
-          (save-excursion
-            (replace-match "")
-            (org-roam-node-insert))
-        (quit (progn
-                (replace-buffer-contents orig)
-                (goto-char p)))))))
-
-(defun org-roam-doctor--remove-link ()
-  "Unlink the text at point."
-  (unless (org-in-regexp org-link-bracket-re 1)
-    (user-error "No link at point"))
-  (save-excursion
-    (let ((label (if (match-end 2)
-                     (match-string-no-properties 2)
-                   (org-link-unescape (match-string-no-properties 1)))))
-      (delete-region (match-beginning 0) (match-end 0))
-      (insert label))))
-
-(defun org-roam-doctor--resolve (msg checker)
-  "Resolve an error.
-MSG is the error that was found, which is displayed in a help buffer.
-CHECKER is a `org-roam-doctor' checker instance."
-  (let ((actions (org-roam-doctor-checker-actions checker))
-        c)
-    (push '("e" . ("Edit" . org-roam-doctor--recursive-edit)) actions)
-    (push '("s" . ("Skip" . org-roam-doctor--skip)) actions)
-    (with-output-to-temp-buffer "*Org-roam-doctor Help*"
-      (mapc #'princ
-            (list "Error message:\n   " msg "\n\n"))
-      (dolist (action actions)
-        (princ (format "[%s]: %s\n"
-                       (car action)
-                       (cadr action))))
-      (princ "\n\n"))
-    (shrink-window-if-larger-than-buffer
-     (get-buffer-window "*Org-roam-doctor Help*"))
-    (message "Press key for command:")
-    (unwind-protect
-        (progn
-          (cl-loop
-           do (setq c (char-to-string (read-char-exclusive)))
-           until (assoc c actions)
-           do (message "Please enter a valid key for command:"))
-          (funcall (cddr (assoc c actions)))
-          (redisplay))
-      (when (get-buffer-window "*Org-roam-doctor Help*")
-        (delete-window (get-buffer-window "*Org-roam-doctor Help*"))
-        (kill-buffer "*Org-roam-doctor Help*")))))
-
-;;;###autoload
-(defun org-roam-doctor (&optional checkall)
-  "Perform a check on the current buffer to ensure cleanliness.
-If CHECKALL, run the check for all Org-roam files."
-  (interactive "P")
-  (let ((files (if checkall
-                   (org-roam--list-all-files)
-                 (unless (org-roam--org-roam-file-p)
-                   (user-error "Not in an org-roam file"))
-                 `(,(buffer-file-name)))))
-    (org-roam-doctor-start files org-roam-doctor--checkers)))
-
-(defun org-roam-doctor-start (files checkers)
-  "Lint FILES using CHECKERS."
-  (save-window-excursion
-    (let ((existing-buffers (org-roam--get-roam-buffers))
-          (org-inhibit-startup org-roam-doctor-inhibit-startup))
-      (org-id-update-id-locations)
-      (dolist (f files)
-        (let ((buf (find-file-noselect f)))
-          (org-roam-doctor--check buf checkers)
-          (unless (memq buf existing-buffers)
-            (save-buffer buf)
-            (kill-buffer buf))))))
-  (org-roam-message "Linting completed."))
-
-(provide 'org-roam-doctor)
-
-;;; org-roam-doctor.el ends here

org-roam-macs.el

@@ -1,86 +0,0 @@
-;;; org-roam-macs.el --- Macros/utility functions -*- coding: utf-8; lexical-binding: t; -*-
-
-;; Copyright © 2020 Jethro Kuan <jethrokuan95@gmail.com>
-
-;; Author: Jethro Kuan <jethrokuan95@gmail.com>
-;; URL: https://github.com/org-roam/org-roam
-;; Keywords: org-mode, roam, convenience
-;; Version: 2.0.0
-;; Package-Requires: ((emacs "26.1") (dash "2.13") (f "0.17.2") (org "9.4") (emacsql "3.0.0") (emacsql-sqlite3 "1.0.2") (magit-section "2.90.1"))
-
-;; This file is NOT part of GNU Emacs.
-
-;; This program is free software; you can redistribute it and/or modify
-;; it under the terms of the GNU General Public License as published by
-;; the Free Software Foundation; either version 3, or (at your option)
-;; any later version.
-;;
-;; This program is distributed in the hope that it will be useful,
-;; but WITHOUT ANY WARRANTY; without even the implied warranty of
-;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-;; GNU General Public License for more details.
-;;
-;; You should have received a copy of the GNU General Public License
-;; along with GNU Emacs; see the file COPYING.  If not, write to the
-;; Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor,
-;; Boston, MA 02110-1301, USA.
-
-;;; Commentary:
-;;
-;; This library implements macros used throughout org-roam.
-;;
-;;; Code:
-(defmacro org-roam-plist-map! (fn plist)
-  "Map FN over PLIST, modifying it in-place."
-  (declare (indent 1))
-  (let ((plist-var (make-symbol "plist"))
-        (k (make-symbol "k"))
-        (v (make-symbol "v")))
-    `(let ((,plist-var (copy-sequence ,plist)))
-       (while ,plist-var
-         (setq ,k (pop ,plist-var))
-         (setq ,v (pop ,plist-var))
-         (setq ,plist (plist-put ,plist ,k (funcall ,fn ,k ,v)))))))
-
-(defmacro org-roam-with-file (file keep-buf-p &rest body)
-  "Execute BODY within FILE.
-If FILE is nil, execute BODY in the current buffer.
-Kills the buffer if KEEP-BUF-P is nil, and FILE is not yet visited."
-  (declare (indent 2) (debug t))
-  `(let* (new-buf
-          (buf (or (and (not ,file)
-                        (current-buffer)) ;If FILE is nil, use current buffer
-                   (find-buffer-visiting ,file) ; If FILE is already visited, find buffer
-                   (progn
-                     (setq new-buf t)
-                     (find-file-noselect ,file)))) ; Else, visit FILE and return buffer
-          res)
-     (with-current-buffer buf
-       (unless (equal major-mode 'org-mode)
-         (delay-mode-hooks (org-mode)))
-       (setq res (progn ,@body))
-       (unless (and new-buf (not ,keep-buf-p))
-         (save-buffer)))
-     (if (and new-buf (not ,keep-buf-p))
-       (when (find-buffer-visiting ,file)
-         (kill-buffer (find-buffer-visiting ,file))))
-     res))
-
-(defmacro org-roam-with-temp-buffer (file &rest body)
-  "Execute BODY within a temp buffer.
-Like `with-temp-buffer', but propagates `org-roam-directory'.
-If FILE, set `default-directory' to FILE's directory and insert its contents."
-  (declare (indent 1) (debug t))
-  (let ((current-org-roam-directory (make-symbol "current-org-roam-directory")))
-    `(let ((,current-org-roam-directory org-roam-directory))
-       (with-temp-buffer
-         (let ((org-roam-directory ,current-org-roam-directory))
-           (delay-mode-hooks (org-mode))
-           (when ,file
-             (insert-file-contents ,file)
-             (setq-local default-directory (file-name-directory ,file)))
-           ,@body)))))
-
-(provide 'org-roam-macs)
-
-;;; org-roam-macs.el ends here

org-roam-migrate.el

@@ -0,0 +1,210 @@
+;;; org-roam-migrate.el --- Migration utilities from v1 to v2 -*- coding: utf-8; lexical-binding: t; -*-
+
+;; Copyright © 2020-2021 Jethro Kuan <jethrokuan95@gmail.com>
+
+;; Author: Jethro Kuan <jethrokuan95@gmail.com>
+;; URL: https://github.com/org-roam/org-roam
+;; Keywords: org-mode, roam, convenience
+;; Version: 2.1.0
+;; Package-Requires: ((emacs "26.1") (dash "2.13") (f "0.17.2") (org "9.4") (emacsql "3.0.0") (emacsql-sqlite "1.0.0") (magit-section "2.90.1"))
+
+;; This file is NOT part of GNU Emacs.
+
+;; This program is free software; you can redistribute it and/or modify
+;; it under the terms of the GNU General Public License as published by
+;; the Free Software Foundation; either version 3, or (at your option)
+;; any later version.
+;;
+;; This program is distributed in the hope that it will be useful,
+;; but WITHOUT ANY WARRANTY; without even the implied warranty of
+;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+;; GNU General Public License for more details.
+;;
+;; You should have received a copy of the GNU General Public License
+;; along with GNU Emacs; see the file COPYING.  If not, write to the
+;; Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor,
+;; Boston, MA 02110-1301, USA.
+
+;;; Commentary:
+;;
+;; This is a special library provided for the v1 users of this package. It's
+;; purpose is to ease the transition from v1 to v2, by providing migration
+;; utilities to convert from v1 notes to v2 nodes.
+;;
+;;; Code:
+(require 'org-roam)
+
+;;; v1 breaking warning
+(defvar org-roam-v2-ack nil
+  "When set to t, won't display the annoying warning message about the upgrade.
+Need to be set before the package is loaded, otherwise won't take
+any affect.")
+
+(unless org-roam-v2-ack
+  (lwarn 'org-roam :error "
+------------------------------------
+WARNING: You're now on Org-roam v2!
+------------------------------------
+
+You may have arrived here from a package upgrade. Please read the
+wiki entry at
+%s
+for an overview of the major changes.
+
+Notes taken in v1 are incompatible with v2, but you can upgrade
+them to the v2 format via a simple command. To migrate your
+notes, first make sure you're on at least Org 9.4 (check with
+C-h v org-version) and set your org-roam-directory to your notes:
+
+(setq org-roam-directory \"path/to/org/files\")
+
+then, run:
+
+  M-x org-roam-migrate-wizard
+
+If you wish to stay on v1, v1 is unfortunately not distributed on
+MELPA. See org-roam/org-roam-v1 on GitHub on how to install v1.
+
+If you've gone through the migration steps (if necessary), and
+know what you're doing set `org-roam-v2-ack' to `t' to disable
+this warning. You can do so by adding:
+
+(setq org-roam-v2-ack t)
+
+To your init file.
+
+"
+         "https://github.com/org-roam/org-roam/wiki/Hitchhiker's-Rough-Guide-to-Org-roam-V2"))
+
+;;; Migration wizard (v1 -> v2)
+;;;###autoload
+(defun org-roam-migrate-wizard ()
+  "Migrate all notes from to be compatible with Org-roam v2.
+1. Convert all notes from v1 format to v2.
+2. Rebuild the cache.
+3. Replace all file links with ID links."
+  (interactive)
+  (when (yes-or-no-p "Org-roam will now convert all your notes from v1 to v2.
+This will take a while. Are you sure you want to do this?")
+    ;; Back up notes
+    (let ((backup-dir (expand-file-name "org-roam.bak"
+                                        (file-name-directory (directory-file-name org-roam-directory)))))
+      (message "Backing up files to %s" backup-dir)
+      (copy-directory org-roam-directory backup-dir))
+
+    ;; Upgrade database to v2
+    (org-roam-db-sync 'force)
+
+    ;; Convert v1 to v2
+    (dolist (f (org-roam-list-files))
+      (org-roam-with-file f nil
+        (org-roam-migrate-v1-to-v2)))
+
+    ;; Rebuild cache
+    (org-roam-db-sync 'force)
+
+    ;;Replace all file links with ID links
+    (dolist (f (org-roam-list-files))
+      (org-roam-with-file f nil
+        (org-roam-migrate-replace-file-links-with-id)
+        (save-buffer)))))
+
+(defun org-roam-migrate-v1-to-v2 ()
+  "Convert the current buffer to v2 format."
+  ;; Create file level ID
+  (org-with-point-at 1
+    (org-id-get-create))
+  ;; Replace roam_key into properties drawer roam_ref
+  (when-let* ((refs (mapcan #'split-string-and-unquote
+                            (cdar (org-collect-keywords '("roam_key"))))))
+    (let ((case-fold-search t))
+      (org-with-point-at 1
+        (dolist (ref refs)
+          (org-roam-ref-add ref))
+        (while (re-search-forward "^#\\+roam_key:" (point-max) t)
+          (beginning-of-line)
+          (kill-line 1)))))
+
+  ;; Replace roam_alias into properties drawer roam_aliases
+  (when-let* ((aliases (mapcan #'split-string-and-unquote
+                               (cdar (org-collect-keywords '("roam_alias"))))))
+    (dolist (alias aliases)
+      (org-roam-alias-add alias)))
+  (let ((case-fold-search t))
+    (org-with-point-at 1
+      (while (re-search-forward "^#\\+roam_alias:" (point-max) t)
+        (beginning-of-line)
+        (kill-line 1))))
+
+  ;; Replace #+roam_tags into #+filetags
+  (org-with-point-at 1
+    (let* ((roam-tags (org-roam-migrate-get-prop-list "ROAM_TAGS"))
+           (file-tags (cl-mapcan (lambda (value)
+                                   (cl-mapcan
+                                    (lambda (k) (org-split-string k ":"))
+                                    (split-string value)))
+                                 (org-roam-migrate-get-prop-list "FILETAGS")))
+           (tags (append roam-tags file-tags))
+           (tags (seq-map (lambda (tag)
+                            (replace-regexp-in-string
+                             "[^[:alnum:]_@#%]"
+                             "_"
+                             tag)) tags))
+           (tags (seq-uniq tags)))
+      (when tags
+        (org-roam-migrate-prop-set "filetags" (org-make-tag-string tags))))
+    (let ((case-fold-search t))
+      (org-with-point-at 1
+        (while (re-search-forward "^#\\+roam_tags:" (point-max) t)
+          (beginning-of-line)
+          (kill-line 1)))))
+  (save-buffer))
+
+(defun org-roam-migrate-get-prop-list (keyword)
+  "Return prop list for KEYWORD."
+  (let ((re (format "^#\\+%s:[ \t]*\\([^\n]+\\)" (upcase keyword)))
+        lst)
+    (goto-char (point-min))
+    (while (re-search-forward re 2048 t)
+      (setq lst (append lst (split-string-and-unquote
+                             (buffer-substring-no-properties
+                              (match-beginning 1) (match-end 1))))))
+    lst))
+
+(defun org-roam-migrate-prop-set (name value)
+  "Set a file property called NAME to VALUE in buffer file.
+If the property is already set, replace its value."
+  (setq name (downcase name))
+  (org-with-point-at 1
+    (let ((case-fold-search t))
+      (if (re-search-forward (concat "^#\\+" name ":\\(.*\\)")
+                             (point-max) t)
+          (replace-match (concat "#+" name ": " value) 'fixedcase)
+        (while (and (not (eobp))
+                    (looking-at "^[#:]"))
+          (if (save-excursion (end-of-line) (eobp))
+              (progn
+                (end-of-line)
+                (insert "\n"))
+            (forward-line)
+            (beginning-of-line)))
+        (insert "#+" name ": " value "\n")))))
+
+(defun org-roam-migrate-replace-file-links-with-id ()
+  "Replace all file: links with ID links in current buffer."
+  (org-with-point-at 1
+    (while (re-search-forward org-link-bracket-re nil t)
+      (let* ((mdata (match-data))
+             (path (match-string 1))
+             (desc (match-string 2)))
+        (when (string-prefix-p "file:" path)
+          (setq path (expand-file-name (substring path 5)))
+          (when-let ((node-id (caar (org-roam-db-query [:select [id] :from nodes
+                                                        :where (= file $s1)
+                                                        :and (= level 0)] path))))
+            (set-match-data mdata)
+            (replace-match (org-link-make-string (concat "id:" node-id)
+                                                 desc) nil t)))))))
+
+(provide 'org-roam-migrate)
+;;; org-roam-migrate.el ends here

org-roam-mode.el

@@ -1,11 +1,12 @@
-;;; org-roam-mode.el --- create and refresh Org-roam buffers -*- lexical-binding: t -*-
+;;; org-roam-mode.el --- Major mode for special Org-roam buffers -*- lexical-binding: t -*-
-;; Copyright © 2020 Jethro Kuan <jethrokuan95@gmail.com>
+
+;; Copyright © 2020-2021 Jethro Kuan <jethrokuan95@gmail.com>
 
 ;; Author: Jethro Kuan <jethrokuan95@gmail.com>
 ;; URL: https://github.com/org-roam/org-roam
 ;; Keywords: org-mode, roam, convenience
-;; Version: 2.0.0
+;; Version: 2.1.0
-;; Package-Requires: ((emacs "26.1") (dash "2.13") (f "0.17.2") (org "9.4") (emacsql "3.0.0") (emacsql-sqlite3 "1.0.2") (magit-section "2.90.1"))
+;; Package-Requires: ((emacs "26.1") (dash "2.13") (f "0.17.2") (org "9.4") (emacsql "3.0.0") (emacsql-sqlite "1.0.0") (magit-section "2.90.1"))
 
 ;; This file is NOT part of GNU Emacs.
 
@@ -26,19 +27,26 @@
 
 ;;; Commentary:
 ;;
-;; This library implements the abstract major-mode `org-roam-mode', from which
+;; This module implements `org-roam-mode', which is a major mode that used by
-;; almost all other Org-roam major-modes derive.
+;; special Org-roam buffers to display various content in a section-like manner
+;; about the nodes and relevant to them information (e.g. backlinks) with which
+;; the user can interact with.
 ;;
 ;;; Code:
-(require 'magit-section)
+(require 'org-roam)
 
-(require 'org-roam-utils)
+;;;; Declarations
+(defvar org-ref-buffer-hacked)
 
-(defvar org-roam-directory)
+;;; Options
-(defvar org-roam-find-file-hook)
+(defcustom org-roam-mode-section-functions (list #'org-roam-backlinks-section
-
+                                                 #'org-roam-reflinks-section)
-(declare-function org-roam--org-file-p "org-roam")
+  "Functions that insert sections in the `org-roam-mode' based buffers.
-(declare-function org-roam-node-at-point "org-roam")
+Each function is called with one argument, which is an
+`org-roam-node' for which the buffer will be constructed for.
+Normally this node is `org-roam-buffer-current-node'."
+  :group 'org-roam
+  :type 'hook)
 
 ;;; Faces
 (defface org-roam-header-line
@@ -120,83 +128,150 @@ and `:slant'."
   "Face for the dimmer part of the widgets."
   :group 'org-roam-faces)
 
-;;; Variables
+;;; Major mode
-(defvar org-roam-current-node nil
-  "The current node at point.")
-
-(defcustom org-roam-mode-sections (list #'org-roam-backlinks-section
-                                        #'org-roam-reflinks-section)
-  "List of functions that insert sections for Org-roam."
-  :group 'org-roam
-  :type '(repeat function))
-
-;;; The mode
 (defvar org-roam-mode-map
   (let ((map (make-sparse-keymap)))
     (set-keymap-parent map magit-section-mode-map)
-    (define-key map [C-return]  'org-roam-visit-thing)
+    (define-key map [C-return]  'org-roam-buffer-visit-thing)
-    (define-key map (kbd "C-m") 'org-roam-visit-thing)
+    (define-key map (kbd "C-m") 'org-roam-buffer-visit-thing)
-    (define-key map [remap revert-buffer] 'org-roam-buffer-render)
+    (define-key map [remap revert-buffer] 'org-roam-buffer-refresh)
     map)
   "Parent keymap for all keymaps of modes derived from `org-roam-mode'.")
 
 (define-derived-mode org-roam-mode magit-section-mode "Org-roam"
-  "Major mode for Org-roam's buffer."
+  "Major mode for displaying relevant information about Org-roam nodes.
+This mode is used by special Org-roam buffers, such as persistent
+`org-roam-buffer' and dedicated Org-roam buffers
+\(`org-roam-buffer-display-dedicated'), which render the
+information in a section-like manner (see
+`org-roam-mode-section-functions'), with which the user can
+interact with."
   :group 'org-roam
   (face-remap-add-relative 'header-line 'org-roam-header-line))
 
-;;; Key functions
+;;; Buffers
-(defun org-roam-visit-thing ()
+(defvar org-roam-buffer-current-node nil
+  "The node for which an `org-roam-mode' based buffer displays its contents.
+This set both, locally and globally. Normally the local value is
+only set in the `org-roam-mode' based buffers, while the global
+value shows the current node in the persistent `org-roam-buffer'.")
+
+(put 'org-roam-buffer-current-node 'permanent-local t)
+
+(defvar org-roam-buffer-current-directory nil
+  "The `org-roam-directory' value of `org-roam-buffer-current-node'.
+Set both, locally and globally in the same way as `org-roam-buffer-current-node'.")
+
+(put 'org-roam-buffer-current-directory 'permanent-local t)
+
+;;;; Library
+(defun org-roam-buffer-visit-thing ()
   "This is a placeholder command.
 Where applicable, section-specific keymaps bind another command
 which visits the thing at point."
   (interactive)
   (user-error "There is no thing at point that could be visited"))
 
-(defun org-roam-buffer-render ()
+(defun org-roam-buffer-file-at-point (&optional assert)
-  "Render the current node at point."
+  "Return the file at point in the current `org-roam-mode' based buffer.
-  (interactive)
+If ASSERT, throw an error."
-  (when (derived-mode-p 'org-roam-mode)
+  (if-let ((file (magit-section-case
-    (let ((inhibit-read-only t))
+                   (org-roam-node-section (org-roam-node-file (oref it node)))
-      (erase-buffer)
+                   (org-roam-grep-section (oref it file))
-      (org-roam-set-header-line-format (org-roam-node-title org-roam-current-node))
+                   (org-roam-preview-section (oref it file))
-      (magit-insert-section (org-roam)
+                   (t (cl-assert (derived-mode-p 'org-roam-mode))))))
-        (magit-insert-heading)
+      file
-        (dolist (fn org-roam-mode-sections)
+    (when assert
-          (funcall fn org-roam-current-node))))))
+      (user-error "No file at point"))))
 
-(defun org-roam-buffer ()
+(defun org-roam-buffer-refresh ()
-  "Launch an Org-roam buffer for the current node at point."
+  "Refresh the contents of the currently selected Org-roam buffer."
   (interactive)
-  (if-let ((node (org-roam-node-at-point)))
+  (cl-assert (derived-mode-p 'org-roam-mode))
-      (progn
+  (save-excursion (org-roam-buffer-render-contents)))
-        (let ((buffer (get-buffer-create
-                       (concat "org-roam: "
-                               (file-relative-name (buffer-file-name) org-roam-directory)))))
-          (with-current-buffer buffer
-            (org-roam-mode)
-            (setq-local org-roam-current-node node)
-            (org-roam-buffer-render))
-          (switch-to-buffer-other-window buffer)))
-    (user-error "No node at point")))
 
-;;; Persistent buffer
+(defun org-roam-buffer-render-contents ()
+  "Recompute and render the contents of an Org-roam buffer.
+Assumes that the current buffer is an `org-roam-mode' based
+buffer."
+  (let ((inhibit-read-only t))
+    (erase-buffer)
+    (org-roam-mode)
+    (setq-local default-directory org-roam-buffer-current-directory)
+    (setq-local org-roam-directory org-roam-buffer-current-directory)
+    (org-roam-buffer-set-header-line-format
+     (org-roam-node-title org-roam-buffer-current-node))
+    (magit-insert-section (org-roam)
+      (magit-insert-heading)
+      (run-hook-with-args 'org-roam-mode-section-functions org-roam-buffer-current-node))
+    (goto-char 0)))
+
+(defun org-roam-buffer-set-header-line-format (string)
+  "Set the header-line using STRING.
+If the `face' property of any part of STRING is already set, then
+that takes precedence. Also pad the left side of STRING so that
+it aligns with the text area."
+  (setq-local header-line-format
+              (concat (propertize " " 'display '(space :align-to 0))
+                      string)))
+
+;;;; Dedicated buffer
+;;;###autoload
+(defun org-roam-buffer-display-dedicated (node)
+  "Launch NODE dedicated Org-roam buffer.
+Unlike the persistent `org-roam-buffer', the contents of this
+buffer won't be automatically changed and will be held in place.
+
+In interactive calls prompt to select NODE, unless called with
+`universal-argument', in which case NODE will be set to
+`org-roam-node-at-point'."
+  (interactive
+   (list (if current-prefix-arg
+             (org-roam-node-at-point 'assert)
+           (org-roam-node-read nil nil nil 'require-match))))
+  (let ((buffer (get-buffer-create (org-roam-buffer--dedicated-name node))))
+    (with-current-buffer buffer
+      (setq-local org-roam-buffer-current-node node)
+      (setq-local org-roam-buffer-current-directory org-roam-directory)
+      (org-roam-buffer-render-contents))
+    (display-buffer buffer)))
+
+(defun org-roam-buffer--dedicated-name (node)
+  "Construct buffer name for NODE dedicated Org-roam buffer."
+  (let ((title (org-roam-node-title node))
+        (filename (file-relative-name (org-roam-node-file node) org-roam-directory)))
+    (format "*org-roam: %s<%s>*" title filename)))
+
+(defun org-roam-buffer-dedicated-p (&optional buffer)
+  "Return t if an Org-roam BUFFER is a node dedicated one.
+See `org-roam-buffer-display-dedicated' for more details.
+If BUFFER is nil, default it to `current-buffer'."
+  (or buffer (setq buffer (current-buffer)))
+  (string-match-p (concat "^" (regexp-quote "*org-roam: "))
+                  (buffer-name buffer)))
+
+;;;; Persistent buffer
 (defvar org-roam-buffer "*org-roam*"
-  "The persistent Org-roam buffer name.")
+  "The persistent Org-roam buffer name. Must be surround with \"*\".
+The content inside of this buffer will be automatically updated
+to the nearest node at point that comes from the current buffer.
+To toggle its display use `org-roam-buffer-toggle' command.")
 
-(defun org-roam-buffer--post-command-h ()
+(defun org-roam-buffer-toggle ()
-  "Reconstructs the Org-roam buffer.
+  "Toggle display of the persistent `org-roam-buffer'."
-This needs to be quick or infrequent, because this is run at
+  (interactive)
-`post-command-hook'.  If REDISPLAY, force an update of
+  (pcase (org-roam-buffer--visibility)
-the Org-roam buffer."
+    ('visible
-  (when (get-buffer-window org-roam-buffer)
+     (progn
-    (when-let ((node (org-roam-node-at-point)))
+       (delete-window (get-buffer-window org-roam-buffer))
-      (unless (equal node org-roam-current-node)
+       (remove-hook 'post-command-hook #'org-roam-buffer--redisplay-h)))
-        (setq org-roam-current-node node)
+    ((or 'exists 'none)
-        (org-roam-buffer-persistent-redisplay)))))
+     (progn
+       (display-buffer (get-buffer-create org-roam-buffer))
+       (org-roam-buffer-persistent-redisplay)))))
 
 (define-inline org-roam-buffer--visibility ()
-  "Return whether the current visibility state of the org-roam buffer.
+  "Return the current visibility state of the persistent `org-roam-buffer'.
 Valid states are 'visible, 'exists and 'none."
   (declare (side-effect-free t))
   (inline-quote
@@ -205,41 +280,205 @@ Valid states are 'visible, 'exists and 'none."
     ((get-buffer org-roam-buffer) 'exists)
     (t 'none))))
 
-(defun org-roam-buffer-toggle ()
-  "Toggle display of the Org-roam buffer."
-  (interactive)
-  (pcase (org-roam-buffer--visibility)
-    ('visible
-     (progn
-       (delete-window (get-buffer-window org-roam-buffer))
-       (remove-hook 'post-command-hook #'org-roam-buffer--post-command-h)))
-    ((or 'exists 'none)
-     (progn
-       (display-buffer (get-buffer-create org-roam-buffer))
-       (setq org-roam-current-node (org-roam-node-at-point))
-       (org-roam-buffer-persistent-redisplay)))))
-
 (defun org-roam-buffer-persistent-redisplay ()
-  "Recompute contents of the persistent Org-roam buffer.
+  "Recompute contents of the persistent `org-roam-buffer'.
-Has no effect when `org-roam-current-node' is nil."
+Has no effect when there's no `org-roam-node-at-point'."
-  (when org-roam-current-node
+  (when-let ((node (org-roam-node-at-point)))
-    (with-current-buffer (get-buffer-create org-roam-buffer)
+    (unless (equal node org-roam-buffer-current-node)
-      (let ((inhibit-read-only t))
+      (setq org-roam-buffer-current-node node
-        (erase-buffer)
+            org-roam-buffer-current-directory org-roam-directory)
-        (org-roam-mode)
+      (with-current-buffer (get-buffer-create org-roam-buffer)
-        (org-roam-set-header-line-format (org-roam-node-title org-roam-current-node))
+        (org-roam-buffer-render-contents)
-        (magit-insert-section (org-roam)
+        (add-hook 'kill-buffer-hook #'org-roam-buffer--persistent-cleanup-h nil t)))))
-          (magit-insert-heading)
-          (dolist (fn org-roam-mode-sections)
-            (funcall fn org-roam-current-node)))))))
 
-(defun org-roam-buffer--redisplay ()
+(defun org-roam-buffer--persistent-cleanup-h ()
-  "."
+  "Clean-up global state thats dedicated for the persistent `org-roam-buffer'."
-  (add-hook 'post-command-hook #'org-roam-buffer--post-command-h nil t))
+  (setq-default org-roam-buffer-current-node nil
+                org-roam-buffer-current-directory nil))
 
-(add-hook 'org-roam-find-file-hook #'org-roam-buffer--redisplay)
+(add-hook 'org-roam-find-file-hook #'org-roam-buffer--setup-redisplay-h)
+(defun org-roam-buffer--setup-redisplay-h ()
+  "Setup automatic redisplay of the persistent `org-roam-buffer'."
+  (add-hook 'post-command-hook #'org-roam-buffer--redisplay-h nil t))
+
+(defun org-roam-buffer--redisplay-h ()
+  "Reconstruct the persistent `org-roam-buffer'.
+This needs to be quick or infrequent, because this designed to
+run at `post-command-hook'."
+  (and (get-buffer-window org-roam-buffer)
+       (org-roam-buffer-persistent-redisplay)))
 
 ;;; Sections
+;;;; Node
+(defvar org-roam-node-map
+  (let ((map (make-sparse-keymap)))
+    (set-keymap-parent map org-roam-mode-map)
+    (define-key map [remap org-roam-buffer-visit-thing] 'org-roam-node-visit)
+    map)
+  "Keymap for `org-roam-node-section's.")
+
+(defclass org-roam-node-section (magit-section)
+  ((keymap :initform 'org-roam-node-map)
+   (node :initform nil))
+  "A `magit-section' used by `org-roam-mode' to outline NODE in its own heading.")
+
+(cl-defun org-roam-node-insert-section (&key source-node point properties)
+  "Insert section for a link from SOURCE-NODE to some other node.
+The other node is normally `org-roam-buffer-current-node'.
+
+SOURCE-NODE is an `org-roam-node' that links or references with
+the other node.
+
+POINT is a character position where the link is located in
+SOURCE-NODE's file.
+
+PROPERTIES (a plist) contains additional information about the
+link.
+
+Despite the name, this function actually inserts 2 sections at
+the same time:
+
+1. `org-roam-node-section' for a heading that describes
+   SOURCE-NODE. Acts as a parent section of the following one.
+
+2. `org-roam-preview-section' for a preview content that comes
+   from SOURCE-NODE's file for the link (that references the
+   other node) at POINT. Acts a child section of the previous
+   one."
+  (magit-insert-section section (org-roam-node-section)
+    (let ((outline (if-let ((outline (plist-get properties :outline)))
+                       (mapconcat #'org-link-display-format outline " > ")
+                     "Top")))
+      (insert (concat (propertize (org-roam-node-title source-node)
+                                  'font-lock-face 'org-roam-title)
+                      (format " (%s)"
+                              (propertize outline 'font-lock-face 'org-roam-olp)))))
+    (magit-insert-heading)
+    (oset section node source-node)
+    (magit-insert-section section (org-roam-preview-section)
+      (insert (org-roam-fontify-like-in-org-mode
+               (org-roam-preview-get-contents (org-roam-node-file source-node) point))
+              "\n")
+      (oset section file (org-roam-node-file source-node))
+      (oset section point point)
+      (insert ?\n))))
+
+;;;; Preview
+(defvar org-roam-preview-map
+  (let ((map (make-sparse-keymap)))
+    (set-keymap-parent map org-roam-mode-map)
+    (define-key map [remap org-roam-buffer-visit-thing] 'org-roam-preview-visit)
+    map)
+  "Keymap for `org-roam-preview-section's.")
+
+(defclass org-roam-preview-section (magit-section)
+  ((keymap :initform 'org-roam-preview-map)
+   (file :initform nil)
+   (point :initform nil))
+  "A `magit-section' used by `org-roam-mode' to contain preview content.
+The preview content comes from FILE, and the link as at POINT.")
+
+(defun org-roam-preview-visit (file point &optional other-window)
+  "Visit FILE at POINT and return the visited buffer.
+With OTHER-WINDOW non-nil do so in another window.
+In interactive calls OTHER-WINDOW is set with
+`universal-argument'."
+  (interactive (list (org-roam-buffer-file-at-point 'assert)
+                     (oref (magit-current-section) point)
+                     current-prefix-arg))
+  (let ((buf (find-file-noselect file))
+        (display-buffer-fn (if other-window
+                               #'switch-to-buffer-other-window
+                             #'pop-to-buffer-same-window)))
+    (funcall display-buffer-fn buf)
+    (with-current-buffer buf
+      (widen)
+      (goto-char point))
+    (when (org-invisible-p) (org-show-context))
+    buf))
+
+(defun org-roam-preview-get-contents (file point)
+  "Get preview content for FILE at POINT."
+  (save-excursion
+    (org-roam-with-temp-buffer file
+      (goto-char point)
+      (let ((elem (org-element-at-point)))
+        ;; We want the parent element always
+        (while (org-element-property :parent elem)
+          (setq elem (org-element-property :parent elem)))
+        (pcase (car elem)
+          ('headline                    ; show subtree
+           (org-roam-preview-get-entry-text (point-marker) most-positive-fixnum))
+          (_
+           (let ((begin (org-element-property :begin elem))
+                 (end (org-element-property :end elem)))
+             (or (string-trim (buffer-substring-no-properties begin end))
+                 (org-element-property :raw-value elem)))))))))
+
+(defun org-roam-preview-get-entry-text (marker n-lines &optional indent)
+  "Extract entry text from MARKER, at most N-LINES lines.
+This will ignore drawers etc, just get the text.
+If INDENT is given, prefix every line with this string."
+  (let (txt ind)
+    (save-excursion
+      (with-current-buffer (marker-buffer marker)
+        (if (not (derived-mode-p 'org-mode))
+            (setq txt "")
+          (org-with-wide-buffer
+           (goto-char marker)
+           (end-of-line 1)
+           (setq txt (buffer-substring
+                      (min (1+ (point)) (point-max))
+                      (progn (outline-next-heading) (point))))
+           (with-temp-buffer
+             (insert txt)
+             (goto-char (point-min))
+             (while (org-activate-links (point-max))
+               (goto-char (match-end 0)))
+             (goto-char (point-min))
+             (while (re-search-forward org-link-bracket-re (point-max) t)
+               (set-text-properties (match-beginning 0) (match-end 0)
+                                    nil))
+             (goto-char (point-min))
+             (while (re-search-forward org-drawer-regexp nil t)
+               (delete-region
+                (match-beginning 0)
+                (progn (re-search-forward
+                        "^[ \t]*:END:.*\n?" nil 'move)
+                       (point))))
+             (goto-char (point-min))
+             (goto-char (point-max))
+             (skip-chars-backward " \t\n")
+             (when (looking-at "[ \t\n]+\\'") (replace-match ""))
+
+             ;; find and remove min common indentation
+             (goto-char (point-min))
+             (untabify (point-min) (point-max))
+             (setq ind (current-indentation))
+             (while (not (eobp))
+               (unless (looking-at "[ \t]*$")
+                 (setq ind (min ind (current-indentation))))
+               (beginning-of-line 2))
+             (goto-char (point-min))
+             (while (not (eobp))
+               (unless (looking-at "[ \t]*$")
+                 (move-to-column ind)
+                 (delete-region (point-at-bol) (point)))
+               (beginning-of-line 2))
+             (goto-char (point-min))
+             (when indent
+               (while (and (not (eobp)) (re-search-forward "^" nil t))
+                 (replace-match indent t t)))
+             (goto-char (point-min))
+             (while (looking-at "[ \t]*\n") (replace-match ""))
+             (goto-char (point-max))
+             (when (> (org-current-line)
+                      n-lines)
+               (org-goto-line (1+ n-lines))
+               (backward-char 1))
+             (setq txt (buffer-substring (point-min) (point))))))))
+    txt))
+
 ;;;; Backlinks
 (cl-defstruct (org-roam-backlink (:constructor org-roam-backlink-create)
                                  (:copier nil))
@@ -340,41 +579,35 @@ Sorts by title."
            :properties (org-roam-reflink-properties reflink)))
         (insert ?\n)))))
 
-;;;; Unlinked references
+;;;; Grep
 (defvar org-roam-grep-map
   (let ((map (make-sparse-keymap)))
     (set-keymap-parent map org-roam-mode-map)
-    (define-key map [remap org-roam-visit-thing] 'org-roam-file-visit)
+    (define-key map [remap org-roam-buffer-visit-thing] 'org-roam-grep-visit)
     map)
   "Keymap for Org-roam grep result sections.")
 
 (defclass org-roam-grep-section (magit-section)
-  ((keymap :initform org-roam-grep-map)
+  ((keymap :initform 'org-roam-grep-map)
    (file :initform nil)
    (row :initform nil)
-   (col :initform nil)))
+   (col :initform nil))
+  "A `magit-section' used by `org-roam-mode' to contain grep output.")
 
-(defun org-roam-file-at-point (&optional assert)
+(defun org-roam-grep-visit (file &optional other-window row col)
-  "Return the file at point.
+  "Visit FILE at row ROW (if any) and column COL (if any). Return the buffer.
-If ASSERT, throw an error."
+With OTHER-WINDOW non-nil (in interactive calls set with
-  (if-let ((file (magit-section-case
+`universal-argument') display the buffer in another window
-                   (org-roam-node-section (org-roam-node-file (oref it node)))
+instead."
-                   (org-roam-grep-section (oref it file))
+  (interactive (list (org-roam-buffer-file-at-point t)
-                   (org-roam-preview-section (oref it file)))))
-      file
-    (when assert
-      (user-error "No file at point"))))
-
-(defun org-roam-file-visit (file &optional other-window row col)
-  "Visits FILE.
-With a prefix argument OTHER-WINDOW, display the buffer in
-another window instead.
-If ROW, move to the row, and if COL move to the COL."
-  (interactive (list (org-roam-file-at-point t)
                      current-prefix-arg
                      (oref (magit-current-section) row)
                      (oref (magit-current-section) col)))
-  (let ((buf (find-file-noselect file)))
+  (let ((buf (find-file-noselect file))
+        (display-buffer-fn (if other-window
+                               #'switch-to-buffer-other-window
+                             #'pop-to-buffer-same-window)))
+    (funcall display-buffer-fn buf)
     (with-current-buffer buf
       (widen)
       (goto-char (point-min))
@@ -382,10 +615,10 @@ If ROW, move to the row, and if COL move to the COL."
         (forward-line (1- row)))
       (when col
         (forward-char (1- col))))
-    (funcall (if other-window
+    (when (org-invisible-p) (org-show-context))
-                 #'switch-to-buffer-other-window
+    buf))
-               #'pop-to-buffer-same-window) buf)))
 
+;;;; Unlinked references
 (defvar org-roam-unlinked-references-result-re
   (rx (group (one-or-more anything))
       ":"
@@ -400,7 +633,7 @@ If ROW, move to the row, and if COL move to the COL."
   "Return the preview line from FILE.
 This is the ROW within FILE."
   (with-temp-buffer
-    (insert-file-contents-literally file)
+    (insert-file-contents file)
     (forward-line (1- row))
     (buffer-substring-no-properties
      (save-excursion
@@ -419,9 +652,9 @@ References from FILE are excluded."
     (let* ((titles (cons (org-roam-node-title node)
                          (org-roam-node-aliases node)))
            (rg-command (concat "rg -o --vimgrep -P -i "
-                               (string-join (mapcar (lambda (glob) (concat "-g " glob))
+                               (mapconcat (lambda (glob) (concat "-g " glob))
-                                                    (org-roam--list-files-search-globs
+                                          (org-roam--list-files-search-globs org-roam-file-extensions)
-                                                     org-roam-file-extensions)) " ")
+                                          " ")
                                (format " '\\[([^[]]++|(?R))*\\]%s' "
                                        (mapconcat (lambda (title)
                                                     (format "|(\\b%s\\b)" (shell-quote-argument title)))

org-roam-protocol.el

@@ -1,107 +0,0 @@
-;;; org-roam-protocol.el --- Protocol handler for roam:// links  -*- coding: utf-8; lexical-binding: t; -*-
-
-;; Copyright © 2020 Jethro Kuan <jethrokuan95@gmail.com>
-;; Author: Jethro Kuan <jethrokuan95@gmail.com>
-;; URL: https://github.com/org-roam/org-roam
-;; Keywords: org-mode, roam, convenience
-;; Version: 2.0.0
-;; Package-Requires: ((emacs "26.1") (dash "2.13") (f "0.17.2") (org "9.4") (emacsql "3.0.0") (emacsql-sqlite3 "1.0.2") (magit-section "2.90.1"))
-
-;; This file is NOT part of GNU Emacs.
-
-;; This program is free software; you can redistribute it and/or modify
-;; it under the terms of the GNU General Public License as published by
-;; the Free Software Foundation; either version 3, or (at your option)
-;; any later version.
-;;
-;; This program is distributed in the hope that it will be useful,
-;; but WITHOUT ANY WARRANTY; without even the implied warranty of
-;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-;; GNU General Public License for more details.
-;;
-;; You should have received a copy of the GNU General Public License
-;; along with GNU Emacs; see the file COPYING.  If not, write to the
-;; Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor,
-;; Boston, MA 02110-1301, USA.
-
-;;; Commentary:
-;;
-;; We extend org-protocol, adding custom Org-roam handlers. The setup
-;; instructions for `org-protocol' can be found in org-protocol.el.
-;;
-;; We define 2 protocols:
-;;
-;; 1. "roam-file": This protocol simply opens the file given by the FILE key
-;; 2. "roam-ref": This protocol creates or opens a note with the given REF
-;;
-;;; Code:
-(require 'org-protocol)
-(require 'org-roam)
-(eval-when-compile
-  (require 'org-roam-macs))
-(require 'ol) ;; for org-link-decode
-
-(defcustom org-roam-protocol-store-links nil
-  "Whether to store links when capturing websites with `org-roam-protocol'."
-  :type 'boolean
-  :group 'org-roam)
-
-;;;; Functions
-(defun org-roam-protocol-open-ref (info)
-  "Process an org-protocol://roam-ref?ref= style url with INFO.
-
-It opens or creates a note with the given ref.
-
-  javascript:location.href = \\='org-protocol://roam-ref?template=r&ref=\\='+ \\
-        encodeURIComponent(location.href) + \\='&title=\\=' + \\
-        encodeURIComponent(document.title) + \\='&body=\\=' + \\
-        encodeURIComponent(window.getSelection())"
-  (unless (plist-get info :ref)
-    (user-error "No ref key provided"))
-  (org-roam-plist-map! (lambda (k v)
-                         (org-link-decode
-                          (if (equal k :ref)
-                              (org-protocol-sanitize-uri v)
-                            v))) info)
-  (when org-roam-protocol-store-links
-    (push (list (plist-get info :ref)
-                (plist-get info :title)) org-stored-links))
-  (org-link-store-props :type (and (string-match org-link-plain-re
-                                                 (plist-get info :ref))
-                                   (match-string 1 (plist-get info :ref)))
-                        :link (plist-get info :ref)
-                        :annotation (org-link-make-string (plist-get info :ref)
-                                                          (or (plist-get info :title)
-                                                              (plist-get info :ref)))
-                        :initial (or (plist-get info :body) ""))
-  (raise-frame)
-  (org-roam-capture-
-   :keys (plist-get info :template)
-   :node (org-roam-node-create :title (plist-get info :title))
-   :info (list :ref (plist-get info :ref)
-               :body (plist-get info :body))
-   :templates org-roam-capture-ref-templates)
-  nil)
-
-(defun org-roam-protocol-open-file (info)
-  "This handler simply opens the file with emacsclient.
-
-INFO is an alist containing additional information passed by the protocol URL.
-It should contain the FILE key, pointing to the path of the file to open.
-
-  Example protocol string:
-
-org-protocol://roam-file?file=/path/to/file.org"
-  (when-let ((file (plist-get info :file)))
-    (raise-frame)
-    (find-file file))
-  nil)
-
-(push '("org-roam-ref"  :protocol "roam-ref"   :function org-roam-protocol-open-ref)
-      org-protocol-protocol-alist)
-(push '("org-roam-file"  :protocol "roam-file"   :function org-roam-protocol-open-file)
-      org-protocol-protocol-alist)
-
-(provide 'org-roam-protocol)
-
-;;; org-roam-protocol.el ends here

org-roam-refile.el

@@ -1,77 +0,0 @@
-;;; org-roam-refile.el --- Refile Org-roam Notes             -*- lexical-binding: t; -*-
-
-;; Copyright © 2020 Jethro Kuan <jethrokuan95@gmail.com>
-
-;; Author: Jethro Kuan <jethrokuan95@gmail.com>
-;; URL: https://github.com/org-roam/org-roam
-;; Keywords: org-mode, roam, convenience
-;; Version: 2.0.0
-;; Package-Requires: ((emacs "26.1") (dash "2.13") (f "0.17.2") (org "9.4") (emacsql "3.0.0") (emacsql-sqlite3 "1.0.2") (magit-section "2.90.1"))
-
-;; This file is NOT part of GNU Emacs.
-
-;; This program is free software; you can redistribute it and/or modify
-;; it under the terms of the GNU General Public License as published by
-;; the Free Software Foundation; either version 3, or (at your option)
-;; any later version.
-;;
-;; This program is distributed in the hope that it will be useful,
-;; but WITHOUT ANY WARRANTY; without even the implied warranty of
-;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-;; GNU General Public License for more details.
-;;
-;; You should have received a copy of the GNU General Public License
-;; along with GNU Emacs; see the file COPYING.  If not, write to the
-;; Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor,
-;; Boston, MA 02110-1301, USA.
-
-;;; Commentary:
-;;
-;; Org-roam refile allows you to refile notes to your nodes.
-;;
-;;; Code:
-(defvar org-auto-align-tags)
-(defvar org-loop-over-headlines-in-active-region)
-
-(defun org-roam-refile ()
-  "Refile to node."
-  (interactive)
-  (let* ((regionp (org-region-active-p))
-         (region-start (and regionp (region-beginning)))
-         (region-end (and regionp (region-end)))
-         (node (org-roam-node-read nil nil 'require-match))
-         (file (org-roam-node-file node))
-         (nbuf (or (find-buffer-visiting file)
-                   (find-file-noselect file)))
-         level reversed)
-    (if regionp
-        (progn
-          (org-kill-new (buffer-substring region-start region-end))
-          (org-save-markers-in-region region-start region-end))
-      (org-copy-subtree 1 nil t))
-    (with-current-buffer nbuf
-      (org-with-wide-buffer
-       (goto-char (org-roam-node-point node))
-       (setq level (org-get-valid-level (funcall outline-level) 1)
-             reversed (org-notes-order-reversed-p))
-       (goto-char
-        (if reversed
-            (or (outline-next-heading) (point-max))
-          (or (save-excursion (org-get-next-sibling))
-              (org-end-of-subtree t t)
-              (point-max))))
-       (unless (bolp) (newline))
-       (org-paste-subtree level nil nil t)
-       (and org-auto-align-tags
-            (let ((org-loop-over-headlines-in-active-region nil))
-              (org-align-tags)))
-       (when (fboundp 'deactivate-mark) (deactivate-mark))))
-    (if regionp
-        (delete-region (point) (+ (point) (- region-end region-start)))
-      (org-preserve-local-variables
-       (delete-region
-        (and (org-back-to-heading t) (point))
-        (min (1+ (buffer-size)) (org-end-of-subtree t t) (point)))))))
-
-(provide 'org-roam-refile)
-;;; org-roam-refile.el ends here

org-roam-utils.el

@@ -1,12 +1,12 @@
-;;; org-roam-utils.el --- Utilities for Org-roam -*- coding: utf-8; lexical-binding: t; -*-
+;;; org-roam-utils.el --- Utilities for Org-roam -*- lexical-binding: t; -*-
 
-;; Copyright © 2020 Jethro Kuan <jethrokuan95@gmail.com>
+;; Copyright © 2020-2021 Jethro Kuan <jethrokuan95@gmail.com>
 
 ;; Author: Jethro Kuan <jethrokuan95@gmail.com>
 ;; URL: https://github.com/org-roam/org-roam
 ;; Keywords: org-mode, roam, convenience
-;; Version: 2.0.0
+;; Version: 2.1.0
-;; Package-Requires: ((emacs "26.1") (dash "2.13") (f "0.17.2") (org "9.4") (emacsql "3.0.0") (emacsql-sqlite3 "1.0.2") (magit-section "2.90.1"))
+;; Package-Requires: ((emacs "26.1") (dash "2.13") (org "9.4"))
 
 ;; This file is NOT part of GNU Emacs.
 
@@ -27,49 +27,112 @@
 
 ;;; Commentary:
 ;;
-;; This library implements utility functions used throughout
+;; This library provides definitions for utilities that used throughout the
-;; Org-roam.
+;; whole package.
-;;
 ;;
 ;;; Code:
-;;;; Library Requires
-(require 'dash)
-(require 's)
 
-(defvar org-roam-verbose)
+;;; String utilities
+;; TODO Refactor this.
+(defun org-roam-replace-string (old new s)
+  "Replace OLD with NEW in S."
+  (declare (pure t) (side-effect-free t))
+  (replace-regexp-in-string (regexp-quote old) new s t t))
 
-;; This is necessary to ensure all dependents on this module see
+(defun org-roam-quote-string (s)
-;; `org-mode-hook' and `org-inhibit-startup' as dynamic variables,
+  "Quotes string S."
-;; regardless of whether Org is loaded before their compilation.
+  (->> s
-(require 'org)
+    (org-roam-replace-string "\\" "\\\\")
+    (org-roam-replace-string "\"" "\\\"")))
 
-;;;; Utility Functions
+;;; List utilities
-(defun org-roam--list-interleave (lst separator)
+(defmacro org-roam-plist-map! (fn plist)
-  "Interleaves elements in LST with SEPARATOR."
+  "Map FN over PLIST, modifying it in-place."
-  (when lst
+  (declare (indent 1))
-    (let ((new-lst (list (pop lst))))
+  (let ((plist-var (make-symbol "plist"))
-      (dolist (it lst)
+        (k (make-symbol "k"))
-        (nconc new-lst (list separator it)))
+        (v (make-symbol "v")))
-      new-lst)))
+    `(let ((,plist-var (copy-sequence ,plist)))
+       (while ,plist-var
+         (setq ,k (pop ,plist-var))
+         (setq ,v (pop ,plist-var))
+         (setq ,plist (plist-put ,plist ,k (funcall ,fn ,k ,v)))))))
 
-(defun org-roam-up-heading-or-point-min ()
+;;; File utilities
-  "Fixed version of Org's `org-up-heading-or-point-min'."
+(defmacro org-roam-with-file (file keep-buf-p &rest body)
-  (ignore-errors (org-back-to-heading t))
+  "Execute BODY within FILE.
-  (let ((p (point)))
+If FILE is nil, execute BODY in the current buffer.
-    (if (< 1 (funcall outline-level))
+Kills the buffer if KEEP-BUF-P is nil, and FILE is not yet visited."
-        (progn
+  (declare (indent 2) (debug t))
-          (org-up-heading-safe)
+  `(let* (new-buf
-          (when (= (point) p)
+          (auto-mode-alist nil)
-            (goto-char (point-min))))
+          (buf (or (and (not ,file)
-      (unless (bobp) (goto-char (point-min))))))
+                        (current-buffer)) ;If FILE is nil, use current buffer
+                   (find-buffer-visiting ,file) ; If FILE is already visited, find buffer
+                   (progn
+                     (setq new-buf t)
+                     (find-file-noselect ,file)))) ; Else, visit FILE and return buffer
+          res)
+     (with-current-buffer buf
+       (unless (equal major-mode 'org-mode)
+         (delay-mode-hooks
+           (let ((org-inhibit-startup t)
+                 (org-agenda-files nil))
+             (org-mode))))
+       (setq res (progn ,@body))
+       (unless (and new-buf (not ,keep-buf-p))
+         (save-buffer)))
+     (if (and new-buf (not ,keep-buf-p))
+         (when (find-buffer-visiting ,file)
+           (kill-buffer (find-buffer-visiting ,file))))
+     res))
 
-(defun org-roam-message (format-string &rest args)
+;;; Buffer utilities
-  "Pass FORMAT-STRING and ARGS to `message' when `org-roam-verbose' is t."
+(defmacro org-roam-with-temp-buffer (file &rest body)
-  (when org-roam-verbose
+  "Execute BODY within a temp buffer.
-    (apply #'message `(,(concat "(org-roam) " format-string) ,@args))))
+Like `with-temp-buffer', but propagates `org-roam-directory'.
+If FILE, set `default-directory' to FILE's directory and insert its contents."
+  (declare (indent 1) (debug t))
+  (let ((current-org-roam-directory (make-symbol "current-org-roam-directory")))
+    `(let ((,current-org-roam-directory org-roam-directory))
+       (with-temp-buffer
+         (let ((org-roam-directory ,current-org-roam-directory))
+           (delay-mode-hooks (org-mode))
+           (when ,file
+             (insert-file-contents ,file)
+             (setq-local default-directory (file-name-directory ,file)))
+           ,@body)))))
 
-(defvar org-ref-buffer-hacked)
+;;; Formatting
+(defun org-roam-format-template (template replacer)
+  "Format TEMPLATE with the function REPLACER.
+The templates are of form ${foo} for variable foo, and
+${foo=default} for variable foo with default value \"default\".
+REPLACER takes an argument of the format variable and the default
+value (possibly nil). Adapted from `s-format'."
+  (let ((saved-match-data (match-data)))
+    (unwind-protect
+        (replace-regexp-in-string
+         "\\${\\([^}]+\\)}"
+         (lambda (md)
+           (let ((var (match-string 1 md))
+                 (replacer-match-data (match-data))
+                 default-val)
+             (when (string-match "\\(.+\\)=\\(.+\\)" var)
+               (setq default-val (match-string 2 var)
+                     var (match-string 1 var)))
+             (unwind-protect
+                 (let ((v (progn
+                            (set-match-data saved-match-data)
+                            (funcall replacer var default-val))))
+                   (if v (format "%s" v) (signal 'org-roam-format-resolve md)))
+               (set-match-data replacer-match-data))))
+         template
+         ;; Need literal to make sure it works
+         t t)
+      (set-match-data saved-match-data))))
 
+;;; Fontification
 (defun org-roam-fontify-like-in-org-mode (s)
   "Fontify string S like in Org mode.
 Like `org-fontify-like-in-org-mode', but supports `org-ref'."
@@ -96,16 +159,7 @@ Like `org-fontify-like-in-org-mode', but supports `org-ref'."
       (org-font-lock-ensure)
       (buffer-string))))
 
-(defun org-roam-set-header-line-format (string)
+;;;; Shielding regions
-  "Set the header-line using STRING.
-If the `face' property of any part of STRING is already set, then
-that takes precedence. Also pad the left side of STRING so that
-it aligns with the text area."
-  (setq-local header-line-format
-              (concat (propertize " " 'display '(space :align-to 0))
-                      string)))
-
-;;; Shielding regions
 (defface org-roam-shielded
   '((t :inherit (warning)))
   "Face for regions that are shielded (marked as read-only).
@@ -132,63 +186,147 @@ BEG and END are markers for the beginning and end regions."
                                              read-only t)
                             (marker-buffer beg))))
 
-;;; Formatting
+;;; Org-mode utilities
-(defun org-roam-format (template replacer)
+;;;; Motions
-  "Format TEMPLATE with the function REPLACER.
+(defun org-roam-up-heading-or-point-min ()
-REPLACER takes an argument of the format variable and optionally
+  "Fixed version of Org's `org-up-heading-or-point-min'."
-an extra argument which is the EXTRA value from the call to
+  (ignore-errors (org-back-to-heading t))
-`org-roam-format'.
+  (let ((p (point)))
-Adapted from `s-format'."
+    (if (< 1 (funcall outline-level))
-  (let ((saved-match-data (match-data)))
+        (progn
-    (unwind-protect
+          (org-up-heading-safe)
-        (replace-regexp-in-string
+          (when (= (point) p)
-         "\\${\\([^}]+\\)}"
+            (goto-char (point-min))))
-         (lambda (md)
+      (unless (bobp) (goto-char (point-min))))))
-           (let ((var (match-string 1 md))
-                 (replacer-match-data (match-data)))
-             (unwind-protect
-                 (let ((v (progn
-                            (set-match-data saved-match-data)
-                            (funcall replacer var))))
-                   (if v (format "%s" v) (signal 'org-roam-format-resolve md)))
-               (set-match-data replacer-match-data)))) template
-         ;; Need literal to make sure it works
-         t t)
-      (set-match-data saved-match-data))))
 
-(defun org-roam--process-display-format (format)
+;;;; Keywords
-  "Pre-calculate minimal widths needed by the FORMAT string."
+(defun org-roam-get-keyword (name &optional file bound)
-  (let* ((fields-width 0)
+  "Return keyword property NAME from an org FILE.
-         (string-width
+FILE defaults to current file.
-          (string-width
+Only scans up to BOUND bytes of the document."
-           (org-roam-format
+  (unless bound
-            format
+    (setq bound 1024))
-            (lambda (field)
+  (if file
-              (setq fields-width
+      (with-temp-buffer
-                    (+ fields-width
+        (insert-file-contents file nil 0 bound)
-                       (string-to-number
+        (org-roam--get-keyword name))
-                        (or (cadr (split-string field ":"))
+    (org-roam--get-keyword name bound)))
-                            "")))))))))
+
-    (cons format (+ fields-width string-width))))
+(defun org-roam--get-keyword (name &optional bound)
+  "Return keyword property NAME in current buffer.
+If BOUND, scan up to BOUND bytes of the buffer."
+  (save-excursion
+    (let ((re (format "^#\\+%s:[ \t]*\\([^\n]+\\)" (upcase name))))
+      (goto-char (point-min))
+      (when (re-search-forward re bound t)
+        (buffer-substring-no-properties (match-beginning 1) (match-end 1))))))
+
+(defun org-roam-set-keyword (key value)
+  "Set keyword KEY to VALUE.
+If the property is already set, it's value is replaced."
+  (org-with-point-at 1
+    (let ((case-fold-search t))
+      (if (re-search-forward (concat "^#\\+" key ":\\(.*\\)") (point-max) t)
+          (if (string-blank-p value)
+              (kill-whole-line)
+            (replace-match (concat " " value) 'fixedcase nil nil 1))
+        (while (and (not (eobp))
+                    (looking-at "^[#:]"))
+          (if (save-excursion (end-of-line) (eobp))
+              (progn
+                (end-of-line)
+                (insert "\n"))
+            (forward-line)
+            (beginning-of-line)))
+        (insert "#+" key ": " value "\n")))))
+
+(defun org-roam-erase-keyword (keyword)
+  "Erase the line where the KEYWORD is, setting line from the top of the file."
+  (let ((case-fold-search t))
+    (org-with-point-at 1
+      (when (re-search-forward (concat "^#\\+" keyword ":") nil t)
+        (beginning-of-line)
+        (delete-region (point) (line-end-position))
+        (delete-char 1)))))
+
+;;;; Properties
+(defun org-roam-add-property (val prop)
+  "Add VAL value to PROP property for the node at point.
+Both, VAL and PROP are strings."
+  (let* ((p (org-entry-get (point) prop))
+         (lst (when p (split-string-and-unquote p)))
+         (lst (if (memq val lst) lst (cons val lst)))
+         (lst (seq-uniq lst)))
+    (org-set-property prop (combine-and-quote-strings lst))
+    val))
+
+(defun org-roam-remove-property (prop &optional val)
+  "Remove VAL value from PROP property for the node at point.
+Both VAL and PROP are strings.
+
+If VAL is not specified, user is prompted to select a value."
+  (let* ((p (org-entry-get (point) prop))
+         (lst (when p (split-string-and-unquote p)))
+         (prop-to-remove (or val (completing-read "Remove: " lst)))
+         (lst (delete prop-to-remove lst)))
+    (if lst
+        (org-set-property prop (combine-and-quote-strings lst))
+      (org-delete-property prop))
+    prop-to-remove))
+
+;;; Logs
+(defvar org-roam-verbose)
+(defun org-roam-message (format-string &rest args)
+  "Pass FORMAT-STRING and ARGS to `message' when `org-roam-verbose' is t."
+  (when org-roam-verbose
+    (apply #'message `(,(concat "(org-roam) " format-string) ,@args))))
 
 ;;; Diagnostics
+;; TODO Update this to also get commit hash
 ;;;###autoload
 (defun org-roam-version (&optional message)
   "Return `org-roam' version.
 Interactively, or when MESSAGE is non-nil, show in the echo area."
   (interactive)
-  (let* ((version
+  (let* ((toplib (or load-file-name buffer-file-name))
-          (with-temp-buffer
+         gitdir topdir version)
-            (insert-file-contents-literally (locate-library "org-roam.el"))
+    (unless (and toplib (equal (file-name-nondirectory toplib) "org-roam-utils.el"))
-            (goto-char (point-min))
+      (setq toplib (locate-library "org-roam-utils.el")))
-            (save-match-data
+    (setq toplib (and toplib (org-roam--straight-chase-links toplib)))
-              (if (re-search-forward "\\(?:;; Version: \\([^z-a]*?$\\)\\)" nil nil)
+    (when toplib
-                  (substring-no-properties (match-string 1))
+      (setq topdir (file-name-directory toplib)
-                "N/A")))))
+            gitdir (expand-file-name ".git" topdir)))
+    (when (file-exists-p gitdir)
+      (setq version
+            (let ((default-directory topdir))
+              (shell-command-to-string "git describe --tags --dirty --always"))))
+    (unless version
+      (setq version (with-temp-buffer
+                      (insert-file-contents-literally (locate-library "org-roam.el"))
+                      (goto-char (point-min))
+                      (save-match-data
+                        (if (re-search-forward "\\(?:;; Version: \\([^z-a]*?$\\)\\)" nil nil)
+                            (substring-no-properties (match-string 1))
+                          "N/A")))))
     (if (or message (called-interactively-p 'interactive))
         (message "%s" version)
       version)))
 
+(defun org-roam--straight-chase-links (filename)
+  "Chase links in FILENAME until a name that is not a link.
+
+This is the same as `file-chase-links', except that it also
+handles fake symlinks that are created by the package manager
+straight.el on Windows.
+
+See <https://github.com/raxod502/straight.el/issues/520>."
+  (when (and (bound-and-true-p straight-symlink-emulation-mode)
+             (fboundp 'straight-chase-emulated-symlink))
+    (when-let ((target (straight-chase-emulated-symlink filename)))
+      (unless (eq target 'broken)
+        (setq filename target))))
+  (file-chase-links filename))
+
 ;;;###autoload
 (defun org-roam-diagnostics ()
   "Collect and print info for `org-roam' issues."
@@ -205,5 +343,6 @@ Interactively, or when MESSAGE is non-nil, show in the echo area."
     (insert (format "- Org: %s\n" (org-version nil 'full)))
     (insert (format "- Org-roam: %s" (org-roam-version)))))
 
+
 (provide 'org-roam-utils)
 ;;; org-roam-utils.el ends here

tests/roam-files/roam-exclude.org

@@ -0,0 +1,7 @@
+:PROPERTIES:
+:ID:       53fadc75-f48e-461e-be06-44a1e88b2abe
+:ROAM_EXCLUDE: t
+:END:
+#+TITLE: Excluded by Org-roam
+
+This node is excluded by declaring ~ROAM_EXCLUDE: t~.

tests/test-org-roam.el

@@ -24,20 +24,43 @@
 (require 'buttercup)
 (require 'org-roam)
 
+(describe "org-roam-list-files"
+  (before-each
+    (setq org-roam-directory (expand-file-name "tests/roam-files")
+          org-roam-db-location (expand-file-name "org-roam.db" temporary-file-directory)
+          org-roam-file-extensions '("org")
+          org-roam-file-exclude-regexp nil))
+
+  (it "gets files correctly"
+    (expect (length (org-roam-list-files))
+            :to-equal 3))
+
+  (it "respects org-roam-file-extensions"
+    (setq org-roam-file-extensions '("md"))
+    (expect (length (org-roam-list-files)) :to-equal 1)
+    (setq org-roam-file-extensions '("org" "md"))
+    (expect (length (org-roam-list-files)) :to-equal 4))
+
+  (it "respects org-roam-file-exclude-regexp"
+    (setq org-roam-file-exclude-regexp (regexp-quote "foo.org"))
+    (expect (length (org-roam-list-files)) :to-equal 2)))
+
 (describe "org-roam-db-sync"
   (before-all
     (setq org-roam-directory (expand-file-name "tests/roam-files")
-          org-roam-db-location (expand-file-name "org-roam.db" temporary-file-directory))
+          org-roam-db-location (expand-file-name "org-roam.db" temporary-file-directory)
-    (org-roam-setup))
+          org-roam-file-extensions '("org")
+          org-roam-file-exclude-regexp nil)
+    (org-roam-db-sync))
 
   (after-all
-    (org-roam-teardown)
+    (org-roam-db--close)
     (delete-file org-roam-db-location))
 
   (it "has the correct number of files"
     (expect (caar (org-roam-db-query [:select (funcall count) :from files]))
             :to-equal
-            2))
+            3))
 
   (it "has the correct number of nodes"
     (expect (caar (org-roam-db-query [:select (funcall count) :from nodes]))
@@ -47,7 +70,13 @@
   (it "has the correct number of links"
     (expect (caar (org-roam-db-query [:select (funcall count) :from links]))
             :to-equal
-            1)))
+            1))
+
+  (it "respects ROAM_EXCLUDE"
+    ;; The excluded node has ID "53fadc75-f48e-461e-be06-44a1e88b2abe"
+    (expect (mapcar #'car (org-roam-db-query [:select id :from nodes]))
+            :to-have-same-items-as
+            '("884b2341-b7fe-434d-848c-5282c0727861" "440795d0-70c1-4165-993d-aebd5eef7a24"))))
 
 (provide 'test-org-roam)