(fix) make roam headline link completion robust (#1473)

Fixes  #1472

.github/workflows/test.yml

@@ -45,10 +45,13 @@ jobs:
 
     - uses: actions/checkout@v2
 
-    - name: Initialize sandbox
+    - name: Create Sandbox Directory
       run: |
         SANDBOX_DIR=$(mktemp -d) || exit 1
-        echo ::set-env name=SANDBOX_DIR::$SANDBOX_DIR
+        echo "SANDBOX_DIR=$SANDBOX_DIR" >> $GITHUB_ENV
+
+    - name: Initialize Sandbox
+      run: |
         ./makem.sh -vv --sandbox $SANDBOX_DIR --install-deps --install-linters
 
     # The "all" rule is not used, because it treats compilation warnings

.gitignore

@@ -12,4 +12,4 @@
 /doc/mimetype
 /doc/stats/
 /config.mk
-/doc/manual/
+/doc/manual.html

CHANGELOG.md

@@ -1,4 +1,54 @@
 # Changelog
+## 1.2.4 (TBD)
+
+### Added
+- [#1396](https://github.com/org-roam/org-roam/pull/1396) add option to choose between prepending, appending, and omitting `roam_tags` in file completion
+- [#1270](https://github.com/org-roam/org-roam/pull/1270) capture: create OLP if it does not exist. Removes need for OLP setup in `:head`.
+- [#1353](https://github.com/org-roam/org-roam/pull/1353) support file-level property drawers
+
+### Changed
+- [#1352](https://github.com/org-roam/org-roam/pull/1352) prefer lower-case for roam_tag and roam_alias in interactive commands
+
+### Fixed
+- [#1281](https://github.com/org-roam/org-roam/pull/1281) fixed idle-timer not instantiated on `org-roam-mode`
+- [#1308](https://github.com/org-roam/org-roam/pull/1308) fixed file renames corrupting database
+- [#1325](https://github.com/org-roam/org-roam/pull/1325) make titles and tags extracted unique per note
+- [#1327](https://github.com/org-roam/org-roam/pull/1327) preserve existing link description during automatic replacement
+- [#1346](https://github.com/org-roam/org-roam/pull/1346) prevent malformed path to `org-roam-index-file`
+- [#1347](https://github.com/org-roam/org-roam/pull/1347) allow use of `%a` element in regular Org-roam captures
+- [#1352](https://github.com/org-roam/org-roam/pull/1352) fixed org-roam-{tag/alias}-{add/delete} altering the original case of the Org property
+- [#1374](https://github.com/org-roam/org-roam/pull/1374) fix headline completions erroring out
+- [#1375](https://github.com/org-roam/org-roam/pull/1375) fix org-roam-protocol to use existing ref file 
+- [#1403](https://github.com/org-roam/org-roam/issues/1403) fixed inconsistency between how we write and read props like alias and tags
+- [#1409](https://github.com/org-roam/org-roam/issues/1398) prevent inclusion of non-org-roam files in `org-roam-dailies--list-files`
+
+## 1.2.3 (13-11-2020)
+
+Primarily a stabilization and bug-fix release.
+
+Org-roam-dailies has also been revamped to include new features, see [this video](https://www.youtube.com/watch?v=1q9x2aZCJJ4) for a quick overview.
+
+### Added
+- [#978](https://github.com/org-roam/org-roam/pull/978) Revamp org-roam-dailies
+- [#1183](https://github.com/org-roam/org-roam/pull/1183) Interactive functions for managing aliases and tags in Org-roam file, namely `org-roam-alias-add`, `org-roam-alias-delete`, `org-roam-tag-add`, and `org-roam-tag-delete`.
+- [#1215](https://github.com/org-roam/org-roam/pull/1215) Multiple `ROAM_KEY` keywords can now be specified in one file. This allows bibliographical entries to share the same note file.
+- [#1238](https://github.com/org-roam/org-roam/pull/1238) Add `org-roam-prefer-id-links` variable to select linking method
+- [#1239](https://github.com/org-roam/org-roam/pull/1239) Allow `org-roam-protocol` to capture the webpage's selection, and add a toggle for storing the links to the pages
+- [#1264](https://github.com/org-roam/org-roam/pull/1264) add `org-roam-db-update-method` to control when the cache is rebuilt.
+
+### Changed
+- [#1264](https://github.com/org-roam/org-roam/pull/1264) renamed `org-roam-update-db-idle-seconds` to `org-roam-db-idle-idle-seconds`
+
+### Fixed
+- [#1074](https://github.com/org-roam/org-roam/issues/1074) fix `org-roam--extract-links` to handle content boundaries.
+- [#1193](https://github.com/org-roam/org-roam/issues/1193) fix `org-roam-db-build-cache` by not killing temporary buffer in `org-roam--extract-links`.
+- [#1195](https://github.com/org-roam/org-roam/issues/1195) fix ID face showing as invalid if within Org ID files, but not Org-roam's.
+- [#1199](https://github.com/org-roam/org-roam/issues/1199) make Org-roam link insertions respect `org-roam-link-title-format` everywhere.
+- [#1201](https://github.com/org-roam/org-roam/issues/1201) fix `org-roam-db-build-cache` failing in scenarios involving duplicate IDs and deleted files.
+- [#1226](https://github.com/org-roam/org-roam/issues/1226) only update relative path of file links
+- [#1232](https://github.com/org-roam/org-roam/issues/1232) fix incorrect title extractions from narrowed buffers
+- [#1233](https://github.com/org-roam/org-roam/issues/1233) fixes bug where descriptive file links become plain links during update for relative paths
+- [#1252](https://github.com/org-roam/org-roam/issues/1252) respect original link type during automatic replacement
 
 ## 1.2.2 (06-10-2020)
 
@@ -31,7 +81,7 @@ This change requires you to set `org-roam-directory` to the resolved path of a f
 - [#974](https://github.com/org-roam/org-roam/pull/974) Protect region targeted by `org-roam-insert`
 - [#994](https://github.com/org-roam/org-roam/pull/994) Simplify org-roam-store-link
 - [#1062](https://github.com/org-roam/org-roam/pull/1062) Variable `org-roam-completions-everywhere` allows for completions everywhere from word at point
-- [#910](https://github.com/org-roam/org-roam/pull/910), [#1105](https://github.com/org-roam/org-roam/pull/1105) Support fuzzy links of the form [[roam:Title]], [[roam:*Headline]] and [[roam:Title*Headline]]
+- [#910](https://github.com/org-roam/org-roam/pull/910), [#1105](https://github.com/org-roam/org-roam/pull/1105) Support fuzzy links of the form `[[roam:Title]]`, `[[roam:*Headline]]` and `[[roam:Title*Headline]]`
 
 ### Bugfixes
 

README.md

@@ -1,47 +1,35 @@
-[![License GPL 3][badge-license]](http://www.gnu.org/licenses/gpl-3.0.txt)
+# Org-roam [![GitHub Release][release-badge]][release] [![MELPA][melpa-badge]][melpa] [![License GPL 3][gpl3-badge]][gpl3]
-[![GitHub Release](https://img.shields.io/github/v/release/org-roam/org-roam)](https://img.shields.io/github/v/release/org-roam/org-roam)
+ 
-[![MELPA](https://melpa.org/packages/org-roam-badge.svg)](https://melpa.org/#/org-roam)
+<img src="https://www.orgroam.com/img/logo.svg" align="right" alt="Org-roam Logo" width="240">
 
-## Synopsis
+Org-roam is a plain-text knowledge management system. It brings some of
+[Roam's][roamresearch] more powerful features into the [Org-mode][org]
+ecosystem.
 
-> **NOTE:** Org-roam builds upon Emacs and Org-mode, both of which are intricate
+Org-roam borrows principles from the Zettelkasten method, providing a solution
-> tools that require time investment for mastery. This makes Org-roam less
+for non-hierarchical note-taking. It should also work as a plug-and-play
-> friendly for beginners, but extremely powerful for those familiar with the
+solution for anyone already using Org-mode for their personal wiki.
-> ecosystem, or willing to invest effort in it.
 
-Org-roam is a [Roam][roamresearch] replica built on top of the
+- **Private and Secure**: Edit your personal wiki completely offline, entirely
-all-powerful [Org-mode][org].
+  in your control. Encrypt your notes with GPG. Take lasting notes in
+  plain-text.
+- **Networked Thought**: Connect notes and thoughts together with ease using
+  backlinks. Discover surprising and previously unseen connections in your notes
+  with the built-in graph visualization.
+- **Extensible and Powerful**: Leverage Emacs' fantastic text-editing interface,
+  and the mature Emacs and Org-mode ecosystem of packages.
+- **Free and Open Source**: Org-roam is licensed under the GNU General Public
+  License version 3 or later.
 
-Org-roam is a solution for effortless non-hierarchical note-taking
+<p align="center">
-with Org-mode. With Org-roam, notes flow naturally, making note-taking
+  <img src="https://www.orgroam.com/img/screenshot.png" alt="Org-roam Screenshot" width="738">
-fun and easy. Org-roam should also work as a plug-and-play solution
+</p>
-for anyone already using Org-mode for their personal wiki.
-
-Org-roam aims to implement the core features of Roam, leveraging the
-mature ecosystem around Org-mode where possible. Eventually, we hope
-to further introduce features enabled by the Emacs ecosystem.
-
-[@technovangelist](https://github.com/technovangelist/) has produced a video
-describing Org-roam and the concepts behind it:
-
-[![Making Connections in your Notes](http://img.youtube.com/vi/Lg61ocfxk3c/0.jpg)](http://www.youtube.com/watch?v=Lg61ocfxk3c "Making Connections in your Notes")
-
-Important links:
 
 - **[Documentation][docs]**
 - **[Discourse][discourse]**
 - **[Slack][slack]**
-
+- **[Frequently Asked Questions][faq]**
-## A Preview
+- **[Changelog](CHANGELOG.md)**
-
-Here's a screencast of Org-roam. The `org-roam-buffer` (window on the
-right) shows backlinks for the active Org-roam buffer (window on the
-left), as well as the surrounding content in the backlink file. The
-database is built once, and updated incrementally. The graph is
-generated from the link structure, and can be used to navigate to the
-respective files.
-
-![img](doc/images/org-roam-graph.gif)
 
 ## Installation
 
@@ -51,7 +39,7 @@ You can install `org-roam` using `package.el`:
 M-x package-install RET org-roam RET
 ```
 
-Here's a sample configuration with using `use-package`:
+Here's a sample configuration with `use-package`:
 
 ```emacs-lisp
 (use-package org-roam
@@ -69,43 +57,30 @@ Here's a sample configuration with using `use-package`:
               (("C-c n I" . org-roam-insert-immediate))))
 ```
 
-`org-roam-graph` by default expects to find the `dot` executable
+Org-roam requires sqlite to function. Org-roam optionally uses Graphviz for
-from the `graphviz` package in the `exec-path`.
+graph-related functionality. It is recommended to install PCRE-enabled ripgrep
-Ensure `graphviz` is installed and found if you want to use this
+for better performance and extended functionality.
-feature or customize your configuration for `org-roam-graph` to use a
-different tool.
-
-For more detailed installation and configuration instructions (including for
-Doom and Spacemacs users), please see [the
-documentation][docs].
-
-## Frequently-asked Questions
-
-Q: How do I create a note whose title already matches one of the candidates (e.g. creating `bar` when `barricade` already exists)?
-
-A: With `ivy`, you need to press `C-M-j` to use the current input instead of the nearest candidate. (Source: [`ivy`’s
-  FAQ](https://github.com/abo-abo/swiper#frequently-asked-questions))
 
 ## Getting Help
 
-Before creating a new topic/issue, please be mindful of our time and ensure
+Before creating a new topic/issue, please be mindful of our time and ensure that
-that it has not already been addressed on
+it has not already been addressed on [GitHub][issues] or on
-[GitHub][issues] or on
 [Discourse][discourse].
 
-- If you are new to Emacs and have problem setting up Org-roam, please ask your question on [Slack, channel #how-do-i][slack].
+- If you are new to Emacs and have problem setting up Org-roam, please ask your
-- For quick questions, please ask them on [Slack, channel #troubleshooting][slack].
+  question on [Slack, channel #how-do-i][slack].
-- If something is not working as it should, or if you would like to suggest a new feature, please [create a new issue][issues].
+- For quick questions, please ask them on [Slack, channel
-- If you have questions about your workflow with the slip-box method, please find a relevant topic on [Discourse][discourse], or create a new one.
+  #troubleshooting][slack].
+- If something is not working as it should, or if you would like to suggest a
+  new feature, please [create a new issue][issues].
+- If you have questions about your workflow with the slip-box method, please
+  find a relevant topic on [Discourse][discourse], or create a new one.
 
 ## Knowledge Bases using Org-roam
 
 - [Jethro Kuan](https://braindump.jethro.dev/)
   ([Source](https://github.com/jethrokuan/braindump/tree/master/org))
-
+- [Alexey Shmalko](https://braindump.rasen.dev/)
-## Changelog
-
-A changelog is being maintained [here](CHANGELOG.md)
 
 ## Contributing
 
@@ -116,12 +91,18 @@ request. Please also see [CONTRIBUTING.md](.github/CONTRIBUTING.md).
 ## License
 
 Copyright © Jethro Kuan and contributors. Distributed under the GNU
-General Public License, Version 3
+General Public License, Version 3.
 
 [roamresearch]: https://www.roamresearch.com/
 [org]: https://orgmode.org/
-[badge-license]: https://img.shields.io/badge/license-GPL_3-green.svg
+[gpl3-badge]: https://img.shields.io/badge/license-GPL_3-green.svg
-[docs]: https://www.orgroam.com/manual/
+[gpl3]: http://www.gnu.org/licenses/gpl-3.0.txt
+[melpa-badge]: https://melpa.org/packages/org-roam-badge.svg
+[melpa]: https://melpa.org/#/org-roam
+[release-badge]: https://img.shields.io/github/v/release/org-roam/org-roam
+[release]: https://github.com/org-roam/org-roam/releases
+[docs]: https://www.orgroam.com/manual.html
 [discourse]: https://org-roam.discourse.group/
 [slack]: https://join.slack.com/t/orgroam/shared_invite/zt-deoqamys-043YQ~s5Tay3iJ5QRI~Lxg
 [issues]: https://github.com/org-roam/org-roam/issues
+[faq]: https://www.orgroam.com/manual.html#FAQ

doc/Makefile

@@ -27,12 +27,9 @@ dir: org-roam.info
 	@printf "Generating $@\n"
 	@$(MAKEINFO) --html --no-split $(MANUAL_HTML_ARGS) $<
 
-html-dir: 
+html-dir:
-	@printf "Generating org-roam/*.html\n"
+	@$(MAKEINFO) --html --no-split $(MANUAL_HTML_ARGS) org-roam.texi
-	@$(MAKEINFO) --html $(MANUAL_HTML_ARGS) org-roam.texi
+	mv org-roam.html manual.html
-	mv org-roam manual
-	cp -r assets manual
-	cp -r images manual
 
 %.pdf: %.texi
 	@printf "Generating $@\n"

doc/assets/page.css

@@ -1,442 +1,68 @@
-/* Import Inter font */
-/* More info at https://github.com/xz/fonts */
-@import url("https://fonts.xz.style/serve/inter.css");
-
 :root {
-  --nc-font-sans: "Inter", -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto,
+  --border: #526980;
-    Oxygen, Ubuntu, Cantarell, "Open Sans", "Helvetica Neue", sans-serif,
+  --code: #007;
-    "Apple Color Emoji", "Segoe UI Emoji", "Segoe UI Symbol";
-  --nc-font-mono: "Courier New", Courier, "Ubuntu Mono", "Liberation Mono",
-    monospace;
-  --nc-tx-1: #000000;
-  --nc-tx-2: #1a1a1a;
-  --nc-bg-1: #ffffff;
-  --nc-bg-2: #f6f8fa;
-  --nc-bg-3: #e5e7eb;
-  --nc-lk-1: #0070f3;
-  --nc-lk-2: #0366d6;
-  --nc-lk-tx: #ffffff;
-  --nc-ac-1: #79ffe1;
-  --nc-ac-tx: #0c4047;
-}
-
-@media (prefers-color-scheme: dark) {
-  :root {
-    --nc-tx-1: #ffffff;
-    --nc-tx-2: #eeeeee;
-    --nc-bg-1: #000000;
-    --nc-bg-2: #111111;
-    --nc-bg-3: #222222;
-    --nc-lk-1: #3291ff;
-    --nc-lk-2: #0070f3;
-    --nc-lk-tx: #ffffff;
-    --nc-ac-1: #7928ca;
-    --nc-ac-tx: #ffffff;
-  }
-}
-
-* {
-  /* Reset margins and padding */
-  margin: 0;
-  padding: 0;
-}
-
-address,
-area,
-article,
-aside,
-audio,
-blockquote,
-datalist,
-details,
-dl,
-fieldset,
-figure,
-form,
-input,
-iframe,
-img,
-meter,
-nav,
-ol,
-optgroup,
-option,
-output,
-p,
-pre,
-progress,
-ruby,
-section,
-table,
-textarea,
-ul,
-video {
-  /* Margins for most elements */
-  margin-bottom: 1rem;
-}
-
-html,
-input,
-select,
-button {
-  /* Set body font family and some finicky elements */
-  font-family: var(--nc-font-sans);
 }
 
 body {
-  /* Center body in page */
+    margin: 5ex 10ex;
-  margin: 0 auto;
+    max-width: 80ex;
-  max-width: 750px;
+    line-height: 1.5;
-  padding: 2rem;
+    font-family: sans-serif;
-  border-radius: 6px;
-  overflow-x: hidden;
-  background: var(--nc-bg-1);
-
-  /* Main body text */
-  color: var(--nc-tx-2);
-  font-size: 1.03rem;
-  line-height: 1.5;
 }
 
-::selection {
+h1, h2, h3 {
-  /* Set background color for selected text */
+    font-weight: normal;
-  background: var(--nc-ac-1);
-  color: var(--nc-ac-tx);
 }
 
-p {
+pre, code {
-  margin-bottom: 1rem;
+    font-family: x, monospace;
-}
-
-h1,
-h2,
-h3,
-h4,
-h5,
-h6 {
-  line-height: 1;
-  color: var(--nc-tx-1);
-  padding-top: 0.875rem;
-}
-
-h1,
-h2,
-h3 {
-  color: var(--nc-tx-1);
-  padding-bottom: 2px;
-  margin-bottom: 8px;
-  border-bottom: 1px solid var(--nc-bg-2);
-}
-
-h4,
-h5,
-h6 {
-  margin-bottom: 0.3rem;
-}
-
-h1 {
-  font-size: 2.25rem;
-}
-
-h2 {
-  font-size: 1.85rem;
-}
-
-h3 {
-  font-size: 1.55rem;
-}
-
-h4 {
-  font-size: 1.25rem;
-}
-
-h5 {
-  font-size: 1rem;
-}
-
-h6 {
-  font-size: 0.875rem;
-}
-
-a {
-  color: var(--nc-lk-1);
-}
-
-a:hover {
-  color: var(--nc-lk-2);
-}
-
-abbr:hover {
-  /* Set the '?' cursor while hovering an abbreviation */
-  cursor: help;
-}
-
-blockquote {
-  padding: 1.5rem;
-  background: var(--nc-bg-2);
-  border-left: 5px solid var(--nc-bg-3);
-}
-
-abbr {
-  cursor: help;
-}
-
-blockquote *:last-child {
-  padding-bottom: 0;
-  margin-bottom: 0;
-}
-
-header {
-  background: var(--nc-bg-2);
-  border-bottom: 1px solid var(--nc-bg-3);
-  padding: 2rem 1.5rem;
-
-  /* This sets the right and left margins to cancel out the body's margins. It's width is still the same, but the background stretches across the page's width. */
-
-  margin: -2rem calc(0px - (50vw - 50%)) 2rem;
-
-  /* Shorthand for:
-
-	margin-top: -2rem;
-	margin-bottom: 2rem;
-
-	margin-left: calc(0px - (50vw - 50%));
-	margin-right: calc(0px - (50vw - 50%)); */
-
-  padding-left: calc(50vw - 50%);
-  padding-right: calc(50vw - 50%);
-}
-
-header h1,
-header h2,
-header h3 {
-  padding-bottom: 0;
-  border-bottom: 0;
-}
-
-header > *:first-child {
-  margin-top: 0;
-  padding-top: 0;
-}
-
-header > *:last-child {
-  margin-bottom: 0;
-}
-
-.button,
-button,
-input[type="submit"],
-input[type="reset"],
-input[type="button"] {
-  font-size: 1rem;
-  display: inline-block;
-  padding: 6px 12px;
-  text-align: center;
-  text-decoration: none;
-  white-space: nowrap;
-  background: var(--nc-lk-1);
-  color: var(--nc-lk-tx);
-  border: 0;
-  border-radius: 4px;
-  box-sizing: border-box;
-  cursor: pointer;
-  color: var(--nc-lk-tx);
-}
-
-.button[disabled],
-button[disabled],
-input[type="submit"][disabled],
-input[type="reset"][disabled],
-input[type="button"][disabled] {
-  cursor: default;
-  opacity: 0.5;
-
-  /* Set the [X] cursor while hovering a disabled link */
-  cursor: not-allowed;
-}
-
-.button:focus,
-.button:hover,
-button:focus,
-button:hover,
-input[type="submit"]:focus,
-input[type="submit"]:hover,
-input[type="reset"]:focus,
-input[type="reset"]:hover,
-input[type="button"]:focus,
-input[type="button"]:hover {
-  background: var(--nc-lk-2);
-}
-
-code,
-pre,
-kbd,
-samp {
-  /* Set the font family for monospaced elements */
-  font-family: var(--nc-font-mono);
-}
-
-code,
-samp,
-kbd,
-pre {
-  /* The main preformatted style. This is changed slightly across different cases. */
-  background: var(--nc-bg-2);
-  border: 1px solid var(--nc-bg-3);
-  border-radius: 4px;
-  padding: 3px 6px;
-  font-size: 0.9rem;
-}
-
-kbd {
-  /* Makes the kbd element look like a keyboard key */
-  border-bottom: 3px solid var(--nc-bg-3);
 }
 
 pre {
-  padding: 1rem 1.4rem;
+    padding: 1ex;
-  max-width: 100%;
+    background: #eee;
-  overflow: auto;
+    border: solid 1px #ddd;
+    min-width: 0;
+    font-size: 80%;
+    overflow: auto;
 }
 
-pre code {
+code {
-  /* When <code> is in a <pre>, reset it's formatting to blend in */
+    color: var(--code);
-  background: inherit;
-  font-size: inherit;
-  color: inherit;
-  border: 0;
-  padding: 0;
-  margin: 0;
-}
-
-code pre {
-  /* When <pre> is in a <code>, reset it's formatting to blend in */
-  display: inline;
-  background: inherit;
-  font-size: inherit;
-  color: inherit;
-  border: 0;
-  padding: 0;
-  margin: 0;
-}
-
-details {
-  /* Make the <details> look more "clickable" */
-  padding: 0.6rem 1rem;
-  background: var(--nc-bg-2);
-  border: 1px solid var(--nc-bg-3);
-  border-radius: 4px;
-}
-
-summary {
-  /* Makes the <summary> look more like a "clickable" link with the pointer cursor */
-  cursor: pointer;
-  font-weight: bold;
-}
-
-details[open] {
-  /* Adjust the <details> padding while open */
-  padding-bottom: 0.75rem;
-}
-
-details[open] summary {
-  /* Adjust the <details> padding while open */
-  margin-bottom: 6px;
-}
-
-details[open] > *:last-child {
-  /* Resets the bottom margin of the last element in the <details> while <details> is opened. This prevents double margins/paddings. */
-  margin-bottom: 0;
-}
-
-dt {
-  font-weight: bold;
-}
-
-dd::before {
-  /* Add an arrow to data table definitions */
-  content: "→ ";
-}
-
-hr {
-  /* Reset the border of the <hr> separator, then set a better line */
-  border: 0;
-  border-bottom: 1px solid var(--nc-bg-3);
-  margin: 1rem auto;
-}
-
-fieldset {
-  margin-top: 1rem;
-  padding: 2rem;
-  border: 1px solid var(--nc-bg-3);
-  border-radius: 4px;
-}
-
-legend {
-  padding: auto 0.5rem;
-}
-
-textarea {
-  /* Don't let the <textarea> extend off the screen naturally or when dragged by the user */
-  max-width: 100%;
-}
-
-ol,
-ul {
-  /* Replace the browser default padding */
-  padding-left: 2rem;
-}
-
-li {
-  margin-top: 0.4rem;
-}
-
-ul ul,
-ol ul,
-ul ol,
-ol ol {
-  margin-bottom: 0;
-}
-
-mark {
-  padding: 3px 6px;
-  background: var(--nc-ac-1);
-  color: var(--nc-ac-tx);
-}
-
-textarea,
-select,
-input {
-  padding: 6px 12px;
-  margin-bottom: 0.5rem;
-  background: var(--nc-bg-2);
-  color: var(--nc-tx-2);
-
-  /* Set a border of the same color as the main background. It isn't visible on idle, but prevents the cell from growing in size when a darker border is set on focus. */
-  border: 1px solid var(--nc-bg-2);
-  border-radius: 4px;
-  box-shadow: none;
-  box-sizing: border-box;
-}
-
-textarea:focus,
-select:focus,
-input[type]:focus {
-  border: 1px solid var(--nc-bg-3);
-
-  /* Reset any browser default outlines */
-  outline: 0;
 }
 
 img {
-  max-width: 100%;
+    max-width: 100%;
 }
 
-/* Customizations */
+table {
-.menu-comment {
+  border-collapse: collapse;
-  font-weight: bold;
+  width: 100%;
-  border: 0;
+}
-  margin: 0;
+
-  padding: 0;
+pre.menu-comment {
-  font-size: 1.2rem;
+  background: none;
+  border: none;
+  font-family: sans-serif;
+  padding: 0;
+  margin: 0;
+  font-size: 100%;
+}
+
+thead {
+  border-bottom: 1px solid var(--border);
+}
+
+tfoot {
+  border-top: 1px solid var(--border);
+}
+
+blockquote {
+  margin-left: 1rem;
+  font-style: italic;
+  font-family: serif;
+  border-left: 3px solid;
+  border-left-color: currentcolor;
+  border-color: var(--text-color);
+  padding-left: 1em;
 }

doc/index.html

@@ -93,7 +93,7 @@
         You can find us on Discourse and Slack.
         
         <ul>
-            <li>Read our documentation within Emacs, or on the <a href="https://www.orgroam.com/manual/">Online Manual</a></li>
+            <li>Read our documentation within Emacs, or on the <a href="https://www.orgroam.com/manual.html">Online Manual</a></li>
             <li>Participate in our forum discussions on <a href="https://org-roam.discourse.group">Discourse</a></li>
             <li>Chat with us on <a href="https://join.slack.com/t/orgroam/shared_invite/zt-deoqamys-043YQ~s5Tay3iJ5QRI~Lxg">Slack</a></li>
           </ul>
@@ -120,13 +120,6 @@
             >org-roam-server</a
           >
         </div>
-        <div class="row">
-          <a
-            class="content footer-links"
-            href="https://github.com/org-roam/company-org-roam"
-            >company-org-roam</a
-          >
-        </div>
       </div>
 
       <div class="col">

org-roam-buffer.el

@@ -5,7 +5,7 @@
 ;; Author: Jethro Kuan <jethrokuan95@gmail.com>
 ;; URL: https://github.com/org-roam/org-roam
 ;; Keywords: org-mode, roam, convenience
-;; Version: 1.2.2
+;; Version: 1.2.3
 ;; Package-Requires: ((emacs "26.1") (dash "2.13") (f "0.17.2") (s "1.12.0") (org "9.3") (emacsql "3.0.0") (emacsql-sqlite3 "1.0.2"))
 
 ;; This file is NOT part of GNU Emacs.
@@ -36,6 +36,8 @@
 (require 's)
 (require 'f)
 (require 'ol)
+(require 'org-element)
+(require 'org-roam-macs)
 
 (defvar org-roam-directory)
 (defvar org-link-frame-setup)
@@ -47,9 +49,10 @@
 (defvar org-roam--org-link-bracket-typed-re)
 
 (declare-function org-roam-db--ensure-built   "org-roam-db")
-(declare-function org-roam--extract-ref       "org-roam")
+(declare-function org-roam-db--get-title      "org-roam-db")
+(declare-function org-roam-db-has-file-p      "org-roam-db")
+(declare-function org-roam--extract-refs      "org-roam")
 (declare-function org-roam--extract-titles    "org-roam")
-(declare-function org-roam--get-title-or-slug "org-roam")
 (declare-function org-roam--get-backlinks     "org-roam")
 (declare-function org-roam-backlinks-mode     "org-roam")
 (declare-function org-roam-mode               "org-roam")
@@ -63,11 +66,13 @@ Valid values are
  * left,
  * right,
  * top,
- * bottom."
+ * bottom,
+ * a function returning one of the above."
   :type '(choice (const left)
                  (const right)
                  (const top)
-                 (const bottom))
+                 (const bottom)
+                 function)
   :group 'org-roam)
 
 (defcustom org-roam-buffer-width 0.33
@@ -95,6 +100,13 @@ Has an effect if and only if `org-roam-buffer-position' is `top' or `bottom'."
   :type 'hook
   :group 'org-roam)
 
+(defcustom org-roam-buffer-preview-function #'org-roam-buffer--preview
+  "Function to obtain preview contents for a given link.
+The function takes in two arguments, the FILE containing the
+link, and the POINT of the link."
+  :type 'function
+  :group 'org-roam)
+
 (defcustom org-roam-buffer-window-parameters nil
   "Additional window parameters for the `org-roam-buffer' side window.
 For example: (setq org-roam-buffer-window-parameters '((no-other-window . t)))"
@@ -107,19 +119,31 @@ For example: (setq org-roam-buffer-window-parameters '((no-other-window . t)))"
 (defun org-roam-buffer--find-file (file)
   "Open FILE in the window `org-roam' was called from."
   (setq file (expand-file-name file))
-  (if (and org-roam-last-window (window-valid-p org-roam-last-window))
+  (let ((last-window org-roam-last-window))
-      (progn (with-selected-window org-roam-last-window
+    (if (window-valid-p last-window)
-               (org-roam--find-file file))
+        (progn (with-selected-window last-window
-             (select-window org-roam-last-window))
+                 (org-roam--find-file file))
-    (org-roam--find-file file)))
+               (select-window last-window))
+      (org-roam--find-file file))))
 
 (defun org-roam-buffer--insert-title ()
   "Insert the org-roam-buffer title."
-  (insert (propertize (org-roam--get-title-or-slug
+  (insert (propertize (org-roam-db--get-title
                        (buffer-file-name org-roam-buffer--current))
                       'font-lock-face
                       'org-document-title)))
 
+(defun org-roam-buffer--preview (file point)
+  "Get preview content for FILE at POINT."
+  (save-excursion
+    (org-roam--with-temp-buffer file
+      (goto-char point)
+      (let ((elem (org-element-at-point)))
+        (or (org-element-property :raw-value elem)
+            (when-let ((begin (org-element-property :begin elem))
+                       (end (org-element-property :end elem)))
+              (string-trim (buffer-substring-no-properties begin end))))))))
+
 (defun org-roam-buffer--pluralize (string number)
   "Conditionally pluralize STRING if NUMBER is above 1."
   (let ((l (pcase number
@@ -148,10 +172,11 @@ ORIG-PATH is the path where the CONTENT originated."
 
 (defun org-roam-buffer--insert-ref-links ()
   "Insert ref backlinks for the current buffer."
-  (when-let ((path (cdr (with-temp-buffer
+  (when-let* ((refs (with-temp-buffer
-                          (insert-buffer-substring org-roam-buffer--current)
+                      (insert-buffer-substring org-roam-buffer--current)
-                          (org-roam--extract-ref)))))
+                      (org-roam--extract-refs)))
-    (if-let* ((key-backlinks (org-roam--get-backlinks path))
+              (paths (mapcar #'cdr refs)))
+    (if-let* ((key-backlinks (mapcan #'org-roam--get-backlinks paths))
               (grouped-backlinks (--group-by (nth 0 it) key-backlinks)))
         (progn
           (insert (let ((l (length key-backlinks)))
@@ -162,60 +187,60 @@ ORIG-PATH is the path where the CONTENT originated."
                   (bls (cdr group)))
               (insert (format "** %s\n"
                               (org-roam-format-link file-from
-                                                     (org-roam--get-title-or-slug file-from)
+                                                    (org-roam-db--get-title file-from)
-                                                     "file")))
+                                                    "file")))
               (dolist (backlink bls)
                 (pcase-let ((`(,file-from _ ,props) backlink))
-                  (insert (propertize (org-roam-buffer-expand-links (plist-get props :content) file-from)
+                  (insert (if-let ((content (funcall org-roam-buffer-preview-function file-from (plist-get props :point))))
-                           'help-echo "mouse-1: visit backlinked note"
+                              (propertize (org-roam-buffer-expand-links content file-from)
-                           'file-from file-from
+                                          'help-echo "mouse-1: visit backlinked note"
-                           'file-from-point (plist-get props :point)))
+                                          'file-from file-from
-                  (insert "\n\n"))))))
+                                          'file-from-point (plist-get props :point))
+                            "")
+                          "\n\n"))))))
       (insert "\n\n* No ref backlinks!"))))
 
 (defun org-roam-buffer--insert-backlinks ()
   "Insert the org-roam-buffer backlinks string for the current buffer."
-  (if-let* ((file-path (buffer-file-name org-roam-buffer--current))
+  (let (props file-from)
-            (titles (with-current-buffer org-roam-buffer--current
+    (if-let* ((file-path (buffer-file-name org-roam-buffer--current))
-                      (org-roam--extract-titles)))
+              (titles (with-current-buffer org-roam-buffer--current
-            (backlinks (org-roam--get-backlinks (push file-path titles)))
+                        (org-roam--extract-titles)))
-            (grouped-backlinks (--group-by (nth 0 it) backlinks)))
+              (backlinks (org-roam--get-backlinks (push file-path titles)))
-      (progn
+              (grouped-backlinks (--group-by (nth 0 it) backlinks)))
-        (insert (let ((l (length backlinks)))
+        (progn
-                     (format "\n\n* %d %s\n"
+          (insert (let ((l (length backlinks)))
-                             l (org-roam-buffer--pluralize "Backlink" l))))
+                    (format "\n\n* %d %s\n"
-        (dolist (group grouped-backlinks)
+                            l (org-roam-buffer--pluralize "Backlink" l))))
-          (let ((file-from (car group))
+          (dolist (group grouped-backlinks)
-                (bls (mapcar (lambda (row)
+            (setq file-from (car group))
-                                 (nth 2 row)) (cdr group))))
+            (setq props (mapcar (lambda (row) (nth 2 row)) (cdr group)))
+            (setq props (seq-sort-by (lambda (p) (plist-get p :point)) #'< props))
             (insert (format "** %s\n"
                             (org-roam-format-link file-from
-                                                   (org-roam--get-title-or-slug file-from)
+                                                  (org-roam-db--get-title file-from)
-                                                   "file")))
+                                                  "file")))
-            ;; Sort backlinks according to time of occurrence in buffer
+            (dolist (prop props)
-            (setq bls (seq-sort-by (lambda (bl)
-                                     (plist-get bl :point))
-                                   #'<
-                                   bls))
-            (dolist (props bls)
               (insert "*** "
-                      (if-let ((outline (plist-get props :outline)))
+                      (if-let ((outline (plist-get prop :outline)))
                           (-> outline
                               (string-join " > ")
                               (org-roam-buffer-expand-links file-from))
                         "Top")
                       "\n"
-                      (propertize
+                      (if-let ((content (funcall org-roam-buffer-preview-function file-from (plist-get prop :point))))
-                       (s-trim (s-replace "\n" " "
+                          (propertize
-                                          (org-roam-buffer-expand-links (plist-get props :content) file-from)))
+                           (s-trim (s-replace "\n" " " (org-roam-buffer-expand-links content file-from)))
-                       'help-echo "mouse-1: visit backlinked note"
+                           'help-echo "mouse-1: visit backlinked note"
-                       'file-from file-from
+                           'file-from file-from
-                       'file-from-point (plist-get props :point))
+                           'file-from-point (plist-get prop :point))
-                      "\n\n")))))
+                        "")
-    (insert "\n\n* No backlinks!")))
+                      "\n\n"))))
+      (insert "\n\n* No backlinks!"))))
 
 (defun org-roam-buffer-update ()
   "Update the `org-roam-buffer'."
+  (interactive)
   (org-roam-db--ensure-built)
   (let* ((source-org-roam-directory org-roam-directory))
     (with-current-buffer org-roam-buffer
@@ -250,7 +275,8 @@ This needs to be quick or infrequent, because this is run at
     (when (and (or redisplay
                    (not (eq org-roam-buffer--current buffer)))
                (eq 'visible (org-roam-buffer--visibility))
-               (buffer-file-name buffer))
+               (buffer-file-name buffer)
+               (org-roam-db-has-file-p (buffer-file-name buffer)))
       (setq org-roam-buffer--current buffer)
       (org-roam-buffer-update))))
 
@@ -289,14 +315,9 @@ Valid states are 'visible, 'exists and 'none."
 
 (defun org-roam-buffer--get-create ()
   "Set up the `org-roam' buffer at `org-roam-buffer-position'."
-  (let ((position
+  (let ((position (if (functionp org-roam-buffer-position)
-         (if (member org-roam-buffer-position '(right left top bottom))
+                      (funcall org-roam-buffer-position)
-             org-roam-buffer-position
+                    org-roam-buffer-position)))
-           (let ((text-quoting-style 'grave))
-             (lwarn '(org-roam) :error
-                    "Invalid org-roam-buffer-position: %s. Defaulting to \\='right"
-                    org-roam-buffer-position))
-           'right)))
     (save-selected-window
       (-> (get-buffer-create org-roam-buffer)
           (display-buffer-in-side-window

org-roam-capture.el

@@ -5,7 +5,7 @@
 ;; Author: Jethro Kuan <jethrokuan95@gmail.com>
 ;; URL: https://github.com/org-roam/org-roam
 ;; Keywords: org-mode, roam, convenience
-;; Version: 1.2.2
+;; Version: 1.2.3
 ;; Package-Requires: ((emacs "26.1") (dash "2.13") (f "0.17.2") (s "1.12.0") (org "9.3") (emacsql "3.0.0") (emacsql-sqlite3 "1.0.2"))
 
 ;; This file is NOT part of GNU Emacs.
@@ -32,6 +32,7 @@
 ;;;; Library Requires
 (require 'org-capture)
 (require 'org-roam-macs)
+(require 'org-roam-db)
 (require 'dash)
 (require 's)
 (require 'cl-lib)
@@ -41,11 +42,13 @@
 (defvar org-roam-directory)
 (defvar org-roam-mode)
 (defvar org-roam-title-to-slug-function)
+(defvar org-roam-file-extensions)
+
 (declare-function  org-roam--get-title-path-completions "org-roam")
 (declare-function  org-roam--get-ref-path-completions   "org-roam")
-(declare-function  org-roam--file-path-from-id          "org-roam")
 (declare-function  org-roam--find-file                  "org-roam")
 (declare-function  org-roam-format-link                "org-roam")
+(declare-function  org-roam--split-ref                 "org-roam")
 (declare-function  org-roam-mode                        "org-roam")
 (declare-function  org-roam-completion--completing-read "org-roam-completion")
 
@@ -74,11 +77,11 @@ note with the given `ref'.")
 (defvar org-roam-capture-additional-template-props nil
   "Additional props to be added to the Org-roam template.")
 
-(defconst org-roam-capture--template-keywords '(:file-name :head)
+(defconst org-roam-capture--template-keywords '(:file-name :head :olp)
   "Keywords used in `org-roam-capture-templates' specific to Org-roam.")
 
 (defcustom org-roam-capture-templates
-  '(("d" "default" plain (function org-roam-capture--get-point)
+  `(("d" "default" plain (function org-roam-capture--get-point)
      "%?"
      :file-name "%<%Y%m%d%H%M%S>-${slug}"
      :head "#+title: ${title}\n"
@@ -218,10 +221,10 @@ Template string   :\n%v")
                  ((const :format "%v " :kill-buffer) (const t))))))
 
 (defcustom org-roam-capture-ref-templates
-  '(("r" "ref" plain (function org-roam-capture--get-point)
+  '(("r" "ref" plain #'org-roam-capture--get-point
      "%?"
      :file-name "${slug}"
-     :head "#+title: ${title}\n#+roam_key: ${ref}\n"
+     :head "#+title: ${title}\n#+roam_key: ${ref}"
      :unnarrowed t))
   "The Org-roam templates used during a capture from the roam-ref protocol.
 Details on how to specify for the template is given in `org-roam-capture-templates'."
@@ -387,13 +390,25 @@ The file is saved if the original value of :no-save is not t and
     (with-current-buffer (org-capture-get :buffer)
       (save-buffer)))))
 
-(defun org-roam-capture--new-file ()
+(defun org-roam-capture--get-file-path (basename)
-  "Return the path to the new file during an Org-roam capture.
+  "Return path for Org-roam file with BASENAME."
+  (let* ((ext (or (car org-roam-file-extensions)
+                  "org"))
+         (file (concat basename "." ext)))
+    (expand-file-name
+     (if org-roam-encrypt-files
+         (concat file ".gpg")
+       file)
+     org-roam-directory)))
+
+(defun org-roam-capture--new-file (&optional allow-existing-file-p)
+  "Return the path to file during an Org-roam capture.
 
 This function reads the file-name attribute of the currently
 active Org-roam template.
 
-If the file path already exists, it throw an error.
+If the file path already exists, and not ALLOW-EXISTING-FILE-P,
+raise a warning.
 
 Else, to insert the header content in the file, `org-capture'
 prepends the `:head' property of the Org-roam capture template.
@@ -402,34 +417,99 @@ To prevent the creation of a new file if the capture process is
 aborted, we do the following:
 
 1. Save the original value of the capture template's :no-save.
-
 2. Set the capture template's :no-save to t.
-
 3. Add a function on `org-capture-before-finalize-hook' that saves
 the file if the original value of :no-save is not t and
 `org-note-abort' is not t."
-  (let* ((name-templ (org-roam-capture--get :file-name))
+  (let* ((name-templ (or (org-roam-capture--get :file-name)
+                         (user-error "Template needs to specify `:file-name'")))
          (new-id (s-trim (org-roam-capture--fill-template
                           name-templ)))
-         (file-path (org-roam--file-path-from-id new-id))
+         (file-path (org-roam-capture--get-file-path new-id))
-         (roam-head (org-roam-capture--get :head))
+         (roam-head (or (org-roam-capture--get :head)
+                        ""))
          (org-template (org-capture-get :template))
          (roam-template (concat roam-head org-template)))
-    (unless (file-exists-p file-path)
+    (if (or (file-exists-p file-path)
+            (find-buffer-visiting file-path))
+        (unless allow-existing-file-p
+          (lwarn '(org-roam) :warning
+                 "Attempted to recreate existing file: %s.
+This can happen when your org-roam db is not in sync with your notes.
+Using existing file..." file-path))
       (make-directory (file-name-directory file-path) t)
       (org-roam-capture--put :orig-no-save (org-capture-get :no-save)
                              :new-file t)
-      (org-capture-put :template
+      (pcase org-roam-capture--context
-                       ;; Fixes org-capture-place-plain-text throwing 'invalid search bound'
+        ('dailies
-                       ;; when both :unnarowed t and "%?" is missing from the template string;
+         ;; Populate the header of the daily file before capture to prevent it
-                       ;; may become unnecessary when the upstream bug is fixed
+         ;; from appearing in the buffer-restriction
-                       (if (s-contains-p "%?" roam-template)
+         (save-window-excursion
-                           roam-template
+           (find-file file-path)
-                         (concat roam-template "%?"))
+           (insert (substring (org-capture-fill-template (concat roam-head "*"))
-                       :type 'plain
+                              0 -2))
-                       :no-save t))
+           (set-buffer-modified-p nil))
+         (org-capture-put :template org-template))
+        (_
+         (org-capture-put :template roam-template
+                          :type 'plain)))
+      (org-capture-put :no-save t))
     file-path))
 
+(defun org-roam-capture-find-or-create-olp (olp)
+  "Return a marker pointing to the entry at OLP in the current buffer.
+If OLP does not exist, create it. If anything goes wrong, throw
+an error, and if you need to do something based on this error,
+you can catch it with `condition-case'."
+  (let* ((level 1)
+         (lmin 1)
+         (lmax 1)
+         (start (point-min))
+         (end (point-max))
+         found flevel)
+    (unless (derived-mode-p 'org-mode)
+      (error "Buffer %s needs to be in Org mode" (current-buffer)))
+    (org-with-wide-buffer
+     (goto-char start)
+     (dolist (heading olp)
+       (let ((re (format org-complex-heading-regexp-format
+                         (regexp-quote heading)))
+             (cnt 0))
+         (while (re-search-forward re end t)
+           (setq level (- (match-end 1) (match-beginning 1)))
+           (when (and (>= level lmin) (<= level lmax))
+             (setq found (match-beginning 0) flevel level cnt (1+ cnt))))
+         (when (> cnt 1)
+           (error "Heading not unique on level %d: %s" lmax heading))
+         (when (= cnt 0)
+           ;; Create heading if it doesn't exist
+           (goto-char end)
+           (unless (bolp) (newline))
+           (org-insert-heading nil nil t)
+           (unless (= lmax 1) (org-do-demote))
+           (insert heading)
+           (setq end (point))
+           (goto-char start)
+           (while (re-search-forward re end t)
+             (setq level (- (match-end 1) (match-beginning 1)))
+             (when (and (>= level lmin) (<= level lmax))
+               (setq found (match-beginning 0) flevel level cnt (1+ cnt))))))
+       (goto-char found)
+       (setq lmin (1+ flevel) lmax (+ lmin (if org-odd-levels-only 1 0)))
+       (setq start found
+             end (save-excursion (org-end-of-subtree t t))))
+     (point-marker))))
+
+(defun org-roam-capture--get-ref-path (type path)
+  "Get the file path to the ref with TYPE and PATH."
+  (caar (org-roam-db-query
+         [:select [file]
+          :from refs
+          :where (= type $s1)
+          :and (= ref $s2)
+          :limit 1]
+         type path)))
+
 (defun org-roam-capture--get-point ()
   "Return exact point to file for org-capture-template.
 The file to use is dependent on the context:
@@ -446,32 +526,45 @@ If there is no file with that ref, a file with that ref is created.
 
 This function is used solely in Org-roam's capture templates: see
 `org-roam-capture-templates'."
-  (let ((file-path (pcase org-roam-capture--context
+  (let* ((file-path (pcase org-roam-capture--context
-                     ('capture
+                      ('capture
-                      (or (cdr (assoc 'file org-roam-capture--info))
+                       (or (cdr (assoc 'file org-roam-capture--info))
-                          (org-roam-capture--new-file)))
+                           (org-roam-capture--new-file)))
-                     ('title
+                      ('title
-                      (org-roam-capture--new-file))
+                       (org-roam-capture--new-file))
-                     ('dailies
+                      ('dailies
-                      (org-capture-put :default-time (cdr (assoc 'time org-roam-capture--info)))
+                       (org-capture-put :default-time (cdr (assoc 'time org-roam-capture--info)))
-                      (org-roam-capture--new-file))
+                       (org-roam-capture--new-file 'allow-existing))
-                     ('ref
+                      ('ref
-                      (let ((completions (org-roam--get-ref-path-completions))
+                       (if-let ((ref (cdr (assoc 'ref org-roam-capture--info))))
-                            (ref (cdr (assoc 'ref org-roam-capture--info))))
+                           (pcase (org-roam--split-ref ref)
-                        (if-let ((pl (cdr (assoc ref completions))))
+                             (`(,type . ,path)
-                            (plist-get pl :path)
+                              (or (org-roam-capture--get-ref-path type path)
-                          (org-roam-capture--new-file))))
+                                  (org-roam-capture--new-file)))
-                     (_ (error "Invalid org-roam-capture-context")))))
+                             (_ (user-error "%s is not a valid ref" ref)))
+                         (error "Ref not found in `org-roam-capture--info'")))
+                      (_ (error "Invalid org-roam-capture-context")))))
     (org-capture-put :template
                      (org-roam-capture--fill-template (org-capture-get :template)))
-    (org-roam-capture--put :file-path file-path)
+    (org-roam-capture--put :file-path file-path
+                           :finalize (or (org-capture-get :finalize)
+                                         (org-roam-capture--get :finalize)))
     (while org-roam-capture-additional-template-props
       (let ((prop (pop org-roam-capture-additional-template-props))
             (val (pop org-roam-capture-additional-template-props)))
         (org-roam-capture--put prop val)))
     (set-buffer (org-capture-target-buffer file-path))
     (widen)
-    (goto-char (point-max))))
+    (if-let* ((olp (org-roam-capture--get :olp)))
+        (condition-case err
+            (when-let ((marker (org-roam-capture-find-or-create-olp olp)))
+              (goto-char marker)
+              (set-marker marker nil))
+          (error
+           (when (org-roam-capture--get :new-file)
+             (kill-buffer))
+           (signal (car err) (cdr err))))
+      (goto-char (point-max)))))
 
 (defun org-roam-capture--convert-template (template)
   "Convert TEMPLATE from Org-roam syntax to `org-capture-templates' syntax."

org-roam-compat.el

@@ -5,7 +5,7 @@
 ;; Author: Jethro Kuan <jethrokuan95@gmail.com>
 ;; URL: https://github.com/org-roam/org-roam
 ;; Keywords: org-mode, roam, convenience
-;; Version: 1.2.2
+;; Version: 1.2.3
 ;; Package-Requires: ((emacs "26.1") (dash "2.13") (f "0.17.2") (s "1.12.0") (org "9.3") (emacsql "3.0.0") (emacsql-sqlite3 "1.0.2"))
 
 ;; This file is NOT part of GNU Emacs.
@@ -75,10 +75,16 @@
   "org-roam 1.1.0")
 (define-obsolete-function-alias 'org-roam--capture 'org-roam-capture--capture
   "org-roam 1.1.0")
-(define-obsolete-function-alias 'org-roam-db--maybe-update 'org-roam-db--update-maybe
-  "org-roam 1.1.0")
 (define-obsolete-function-alias 'org-roam-db--clear 'org-roam-db-clear
   "org-roam 1.2.0")
+(define-obsolete-function-alias 'org-roam-dailies-today 'org-roam-dailies-find-today
+  "org-roam 1.2.2")
+(define-obsolete-function-alias 'org-roam-dailies-yesterday 'org-roam-dailies-find-yesterday
+  "org-roam 1.2.2")
+(define-obsolete-function-alias 'org-roam-dailies-tomorrow 'org-roam-dailies-find-tomorrow
+  "org-roam 1.2.2")
+(define-obsolete-function-alias 'org-roam-dailies-date 'org-roam-dailies-find-date
+  "org-roam 1.2.2")
 
 ;;;; Variables
 (define-obsolete-variable-alias 'org-roam-graphviz-extra-options
@@ -95,6 +101,9 @@
   'org-roam-dailies-capture-templates "org-roam 1.0.0")
 (define-obsolete-variable-alias 'org-roam-date-filename-format
   'org-roam-dailies-capture-templates "org-roam 1.0.0")
+(define-obsolete-variable-alias 'org-roam-update-db-idle-seconds
+  'org-roam-db-update-idle-seconds "org-roam 1.2.2")
+
 (make-obsolete-variable 'org-roam-buffer-no-delete-other-windows
   'org-roam-buffer-window-parameters "org-roam 1.1.1")
 

org-roam-completion.el

@@ -5,7 +5,7 @@
 ;; Author: Jethro Kuan <jethrokuan95@gmail.com>
 ;; URL: https://github.com/org-roam/org-roam
 ;; Keywords: org-mode, roam, convenience
-;; Version: 1.2.2
+;; Version: 1.2.3
 ;; Package-Requires: ((emacs "26.1") (dash "2.13") (f "0.17.2") (s "1.12.0") (org "9.3") (emacsql "3.0.0") (emacsql-sqlite3 "1.0.2"))
 
 ;; This file is NOT part of GNU Emacs.
@@ -79,6 +79,7 @@ Return user choice."
             (if (fboundp 'ivy-read)
                 (ivy-read prompt choices
                           :initial-input initial-input
+                          :preselect initial-input
                           :require-match require-match
                           :action (prog1 action
                                     (setq action nil))

org-roam-dailies.el

@@ -1,11 +1,13 @@
-;;; org-roam-dailies.el --- Daily notes for Org-roam -*- coding: utf-8; lexical-binding: t; -*-
+;;; org-roam-dailies.el --- Daily-notes for Org-roam -*- coding: utf-8; lexical-binding: t; -*-
 ;;;
 ;; Copyright © 2020 Jethro Kuan <jethrokuan95@gmail.com>
+;; Copyright © 2020 Leo Vivier <leo.vivier+dev@gmail.com>
 
 ;; Author: Jethro Kuan <jethrokuan95@gmail.com>
+;; 	Leo Vivier <leo.vivier+dev@gmail.com>
 ;; URL: https://github.com/org-roam/org-roam
 ;; Keywords: org-mode, roam, convenience
-;; Version: 1.2.2
+;; Version: 1.2.3
 ;; Package-Requires: ((emacs "26.1") (dash "2.13") (f "0.17.2") (s "1.12.0") (org "9.3") (emacsql "3.0.0") (emacsql-sqlite3 "1.0.2"))
 
 ;; This file is NOT part of GNU Emacs.
@@ -27,7 +29,7 @@
 
 ;;; Commentary:
 ;;
-;; This library provides functionality for creating daily notes. This is a
+;; This library provides functionality for creating daily-notes. This is a
 ;; concept borrowed from Roam Research.
 ;;
 ;;; Code:
@@ -35,102 +37,311 @@
 (require 'org-capture)
 (require 'org-roam-capture)
 (require 'org-roam-macs)
+(require 'f)
+
+;;;; Declarations
+(defvar org-roam-mode)
+(defvar org-roam-directory)
+(defvar org-roam-file-extensions)
+(declare-function org-roam--org-file-p        "org-roam")
+(declare-function org-roam--find-file         "org-roam")
+(declare-function org-roam-mode               "org-roam")
+
+;;;; Customizable variables
+(defcustom org-roam-dailies-directory "daily/"
+  "Path to daily-notes."
+  :group 'org-roam
+  :type 'string)
+
+(defcustom org-roam-dailies-find-file-hook nil
+  "Hook that is run right after navigating to a daily-note."
+  :group 'org-roam
+  :type 'hook)
 
 (defcustom org-roam-dailies-capture-templates
-  '(("d" "daily" plain (function org-roam-capture--get-point)
+  `(("d" "default" entry (function org-roam-capture--get-point)
-     ""
+     "* %?"
-     :immediate-finish t
+     :file-name ,(concat org-roam-dailies-directory "%<%Y-%m-%d>")
-     :file-name "%<%Y-%m-%d>"
+     :head "#+title: %<%Y-%m-%d>\n"))
-     :head "#+title: %<%Y-%m-%d>"))
+  "Capture templates for daily-notes in Org-roam."
-  "Capture templates for daily notes in Org-roam."
   :group 'org-roam
   ;; Adapted from `org-capture-templates'
   :type
-  '(repeat
+  `(repeat
-    (choice :value ("d" "daily" plain (function org-roam-capture--get-point)
+    (choice :value ("d" "default" plain (function org-roam-capture--get-point)
-                    ""
+                    "%?"
-                    :immediate-finish t
+                    :file-name ,(concat org-roam-dailies-directory
-                    :file-name "%<%Y-%m-%d>"
+					"%<%Y-%m-%d>")
-                    :head "#+title: %<%Y-%m-%d>")
+                    :head "#+title: %<%Y-%m-%d>\n"
-            (list :tag "Multikey description"
+                    :unnarrowed t)
-                  (string :tag "Keys       ")
+      (list :tag "Multikey description"
-                  (string :tag "Description"))
+        (string :tag "Keys       ")
-            (list :tag "Template entry"
+        (string :tag "Description"))
-                  (string :tag "Keys              ")
+      (list :tag "Template entry"
-                  (string :tag "Description       ")
+        (string :tag "Keys              ")
-                  (const :format "" plain)
+        (string :tag "Description       ")
-                  (const :format "" (function org-roam-capture--get-point))
+        (choice :tag "Type              "
-                  (choice :tag "Template          "
+         (const :tag "Plain" plain)
-                          (string :tag "String"
+         (const :tag "Entry (for creating headlines)" entry))
-                                  :format "String:\n            \
+        (const :format "" #'org-roam-capture--get-point)
+        (choice :tag "Template          "
+          (string :tag "String"
+                  :format "String:\n            \
 Template string   :\n%v")
-                          (list :tag "File"
+          (list :tag "File"
-                                (const :format "" file)
+            (const :format "" file)
-                                (file :tag "Template file     "))
+            (file :tag "Template file     "))
-                          (list :tag "Function"
+          (list :tag "Function"
-                                (const :format "" function)
+            (const :format "" function)
-                                (function :tag "Template function ")))
+            (function :tag "Template function ")))
-                  (const :format "" :immediate-finish) (const :format "" t)
+        (const :format "File name format  :" :file-name)
-                  (const :format "File name format  :" :file-name)
+        (string :format " %v" :value ,(concat org-roam-dailies-directory
-                  (string :format " %v" :value "#+title: ${title}\n")
+					      "%<%Y-%m-%d>"))
-                  (const :format "Header format     :" :head)
+        (const :format "Header format     :" :head)
-                  (string :format "\n%v" :value "%<%Y%m%d%H%M%S>-${slug}")
+        (string :format " %v" :value "#+title: ${title}\n")
-                  (plist :inline t
+        (plist :inline t
-                         :tag "Options"
+               :tag "Options"
-                         ;; Give the most common options as checkboxes
+          ;; Give the most common options as checkboxes
-                         :options
+               :options
-                         (((const :format "%v " :prepend) (const t))
+               (((const :tag "Outline path" :olp)
-                          ((const :format "%v " :jump-to-captured) (const t))
+                 (repeat :tag "Headings"
-                          ((const :format "%v " :empty-lines) (const 1))
+                   (string :tag "Heading")))
-                          ((const :format "%v " :empty-lines-before) (const 1))
+                ((const :format "%v " :unnarrowed) (const t))
-                          ((const :format "%v " :empty-lines-after) (const 1))
+                ((const :format "%v " :prepend) (const t))
-                          ((const :format "%v " :clock-in) (const t))
+                ((const :format "%v " :immediate-finish) (const t))
-                          ((const :format "%v " :clock-keep) (const t))
+                ((const :format "%v " :jump-to-captured) (const t))
-                          ((const :format "%v " :clock-resume) (const t))
+                ((const :format "%v " :empty-lines) (const 1))
-                          ((const :format "%v " :time-prompt) (const t))
+                ((const :format "%v " :empty-lines-before) (const 1))
-                          ((const :format "%v " :tree-type) (const week))
+                ((const :format "%v " :empty-lines-after) (const 1))
-                          ((const :format "%v " :table-line-pos) (string))
+                ((const :format "%v " :clock-in) (const t))
-                          ((const :format "%v " :kill-buffer) (const t))
+                ((const :format "%v " :clock-keep) (const t))
-                          ((const :format "%v " :unnarrowed) (const t))))))))
+                ((const :format "%v " :clock-resume) (const t))
+                ((const :format "%v " :time-prompt) (const t))
+                ((const :format "%v " :tree-type) (const week))
+                ((const :format "%v " :table-line-pos) (string))
+                ((const :format "%v " :kill-buffer) (const t))))))))
 
-;; Declarations
+;;;; Utilities
-(defvar org-roam-mode)
+(defun org-roam-dailies-directory--get-absolute-path ()
-(declare-function org-roam--file-path-from-id "org-roam")
+  "Get absolute path to `org-roam-dailies-directory'."
-(declare-function org-roam-mode               "org-roam")
+  (expand-file-name org-roam-dailies-directory org-roam-directory))
 
-(defun org-roam-dailies--file-for-time (time)
+(defun org-roam-dailies-find-directory ()
-  "Create and find file for TIME."
+  "Find and open `org-roam-dailies-directory'."
-  (let ((org-roam-capture-templates org-roam-dailies-capture-templates)
+  (interactive)
+  (org-roam--find-file (org-roam-dailies-directory--get-absolute-path)))
+
+(defun org-roam-dailies--daily-note-p (&optional file)
+  "Return t if FILE is an Org-roam daily-note, nil otherwise.
+
+If FILE is not specified, use the current buffer's file-path."
+  (when-let ((path (or file
+                       (-> (buffer-base-buffer)
+                           (buffer-file-name))))
+             (directory (org-roam-dailies-directory--get-absolute-path)))
+    (setq path (expand-file-name path))
+    (save-match-data
+      (and
+       (org-roam--org-file-p path)
+       (f-descendant-of-p path directory)))))
+
+(defun org-roam-dailies--capture (time &optional goto)
+  "Capture an entry in a daily-note for TIME, creating it if necessary.
+
+When GOTO is non-nil, go the note without creating an entry."
+  (unless org-roam-mode (org-roam-mode))
+  (let ((org-roam-capture-templates (--> org-roam-dailies-capture-templates
+                                         (if goto (list (car it)) it)))
         (org-roam-capture--info (list (cons 'time time)))
         (org-roam-capture--context 'dailies))
-    (setq org-roam-capture-additional-template-props (list :finalize 'find-file))
+    (org-roam-capture--capture (when goto '(4)))))
-    (org-roam-capture--capture)))
 
-(defun org-roam-dailies-today ()
+;;;; Commands
-  "Create and find the daily note for today."
+;;; Today
+(defun org-roam-dailies-capture-today (&optional goto)
+  "Create an entry in the daily-note for today.
+
+When GOTO is non-nil, go the note without creating an entry."
+  (interactive "P")
+  (org-roam-dailies--capture (current-time) goto)
+  (when goto
+    (run-hooks 'org-roam-dailies-find-file-hook)))
+
+(defun org-roam-dailies-find-today ()
+  "Find the daily-note for today, creating it if necessary."
   (interactive)
-  (unless org-roam-mode (org-roam-mode))
+  (org-roam-dailies-capture-today t))
-  (org-roam-dailies--file-for-time (current-time)))
 
-(defun org-roam-dailies-tomorrow (n)
+;;; Tomorrow
-  "Create and find the daily note for tomorrow.
+(defun org-roam-dailies-capture-tomorrow (n &optional goto)
-With numeric argument N, use N days in the future."
+  "Create an entry in the daily-note for tomorrow.
+
+With numeric argument N, use the daily-note N days in the future.
+
+With a `C-u' prefix or when GOTO is non-nil, go the note without
+creating an entry."
   (interactive "p")
-  (unless org-roam-mode (org-roam-mode))
+  (org-roam-dailies--capture (time-add (* n 86400) (current-time)) goto))
-  (org-roam-dailies--file-for-time (time-add (* n 86400) (current-time))))
 
-(defun org-roam-dailies-yesterday (n)
+(defun org-roam-dailies-find-tomorrow (n)
-  "Create and find the file for yesterday.
+  "Find the daily-note for tomorrow, creating it if necessary.
-With numeric argument N, use N days in the past."
+
+With numeric argument N, use the daily-note N days in the
+future."
   (interactive "p")
-  (unless org-roam-mode (org-roam-mode))
+  (org-roam-dailies-capture-tomorrow n t))
-  (org-roam-dailies-tomorrow (- n)))
 
-(defun org-roam-dailies-date ()
+;;; Yesterday
-  "Create the file for any date using the calendar interface."
+(defun org-roam-dailies-capture-yesterday (n &optional goto)
+  "Create an entry in the daily-note for yesteday.
+
+With numeric argument N, use the daily-note N days in the past.
+
+When GOTO is non-nil, go the note without creating an entry."
+  (interactive "p")
+  (org-roam-dailies-capture-tomorrow (- n) goto))
+
+(defun org-roam-dailies-find-yesterday (n)
+  "Find the daily-note for yesterday, creating it if necessary.
+
+With numeric argument N, use the daily-note N days in the
+future."
+  (interactive "p")
+  (org-roam-dailies-capture-tomorrow (- n) t))
+
+;;; Calendar
+(defvar org-roam-dailies-calendar-hook (list 'org-roam-dailies-calendar-mark-entries)
+  "Hooks to run when showing the `org-roam-dailies-calendar'.")
+
+(defun org-roam-dailies-calendar--install-hook ()
+  "Install Org-roam-dailies hooks to calendar."
+  (add-hook 'calendar-today-visible-hook #'org-roam-dailies-calendar--run-hook)
+  (add-hook 'calendar-today-invisible-hook #'org-roam-dailies-calendar--run-hook))
+
+(defun org-roam-dailies-calendar--run-hook ()
+  "Run Org-roam-dailies hooks to calendar."
+  (run-hooks 'org-roam-dailies-calendar-hook)
+  (remove-hook 'calendar-today-visible-hook #'org-roam-dailies-calendar--run-hook)
+  (remove-hook 'calendar-today-invisible-hook #'org-roam-dailies-calendar--run-hook))
+
+(defun org-roam-dailies-calendar--file-to-date (&optional file)
+  "Convert FILE to date.
+
+Return (MONTH DAY YEAR)."
+  (let ((file (or file
+                  (-> (buffer-base-buffer)
+                      (buffer-file-name)))))
+    (cl-destructuring-bind (_ _ _ d m y _ _ _)
+        (-> file
+            (file-name-nondirectory)
+            (file-name-sans-extension)
+            (org-parse-time-string))
+      (list m d y))))
+
+(defun org-roam-dailies-calendar--date-to-time (date)
+  "Convert DATE as returned from the calendar (MONTH DAY YEAR) to a time."
+  (encode-time 0 0 0 (nth 1 date) (nth 0 date) (nth 2 date)))
+
+(defun org-roam-dailies-calendar-mark-entries ()
+  "Mark days in the calendar for which a daily-note is present."
+  (when (file-exists-p (org-roam-dailies-directory--get-absolute-path))
+    (dolist (date (mapcar #'org-roam-dailies-calendar--file-to-date
+                          (org-roam-dailies--list-files)))
+      (when (calendar-date-is-visible-p date)
+        (calendar-mark-visible-date date 'org-roam-dailies-calendar-note)))))
+
+;;; Date
+(defun org-roam-dailies-capture-date (&optional goto prefer-future)
+  "Create an entry in the daily-note for a date using the calendar.
+
+Prefer past dates, unless PREFER-FUTURE is non-nil.
+
+With a `C-u' prefix or when GOTO is non-nil, go the note without
+creating an entry."
+  (interactive "P")
+  (org-roam-dailies-calendar--install-hook)
+  (let* ((time-str (let ((org-read-date-prefer-future prefer-future))
+                     (org-read-date nil nil nil (if goto
+                                                    "Find daily-note: "
+                                                  "Capture to daily-note: "))))
+         (time (org-read-date nil t time-str)))
+    (org-roam-dailies--capture time goto)
+    (when goto
+      (run-hooks 'org-roam-dailies-find-file-hook))))
+
+(defun org-roam-dailies-find-date (&optional prefer-future)
+  "Find the daily-note for a date using the calendar, creating it if necessary.
+
+Prefer past dates, unless PREFER-FUTURE is non-nil."
   (interactive)
-  (let ((time (org-read-date nil 'to-time nil "Date:  ")))
+  (org-roam-dailies-capture-date t prefer-future))
-    (org-roam-dailies--file-for-time time)))
+
+;;; Navigation
+(defun org-roam-dailies--list-files (&rest extra-files)
+  "List all files in `org-roam-dailies-directory'.
+EXTRA-FILES can be used to append extra files to the list."
+  (let ((dir (org-roam-dailies-directory--get-absolute-path))
+	(regexp (rx-to-string `(and "." (or ,@org-roam-file-extensions)))))
+    (append (--remove (let ((file (file-name-nondirectory it)))
+                        (when (or (auto-save-file-name-p file)
+                                  (backup-file-name-p file)
+                                  (string-match "^\\." file))
+                          it))
+                      (directory-files-recursively dir regexp))
+            extra-files)))
+
+(defun org-roam-dailies-find-next-note (&optional n)
+  "Find next daily-note.
+
+With numeric argument N, find note N days in the future. If N is
+negative, find note N days in the past."
+  (interactive "p")
+  (unless (org-roam-dailies--daily-note-p)
+    (user-error "Not in a daily-note"))
+  (setq n (or n 1))
+  (let* ((dailies (org-roam-dailies--list-files))
+         (position
+          (cl-position-if (lambda (candidate)
+                            (string= (buffer-file-name (buffer-base-buffer)) candidate))
+                          dailies))
+         note)
+      (unless position
+        (user-error "Can't find current note file - have you saved it yet?"))
+      (pcase n
+        ((pred (natnump))
+         (when (eq position (- (length dailies) 1))
+           (user-error "Already at newest note")))
+        ((pred (integerp))
+         (when (eq position 0)
+             (user-error "Already at oldest note"))))
+      (setq note (nth (+ position n) dailies))
+      (find-file note)
+      (run-hooks 'org-roam-dailies-find-file-hook)))
+
+(defun org-roam-dailies-find-previous-note (&optional n)
+  "Find previous daily-note.
+
+With numeric argument N, find note N days in the past. If N is
+negative, find note N days in the future."
+  (interactive "p")
+  (let ((n (if n (- n) -1)))
+    (org-roam-dailies-find-next-note n)))
+
+;;;; Bindings
+(defvar org-roam-dailies-map (make-sparse-keymap)
+  "Keymap for `org-roam-dailies'.")
+
+(define-prefix-command 'org-roam-dailies-map)
+
+(define-key org-roam-dailies-map (kbd "d") #'org-roam-dailies-find-today)
+(define-key org-roam-dailies-map (kbd "y") #'org-roam-dailies-find-yesterday)
+(define-key org-roam-dailies-map (kbd "t") #'org-roam-dailies-find-tomorrow)
+(define-key org-roam-dailies-map (kbd "n") #'org-roam-dailies-capture-today)
+(define-key org-roam-dailies-map (kbd "f") #'org-roam-dailies-find-next-note)
+(define-key org-roam-dailies-map (kbd "b") #'org-roam-dailies-find-previous-note)
+(define-key org-roam-dailies-map (kbd "c") #'org-roam-dailies-find-date)
+(define-key org-roam-dailies-map (kbd "v") #'org-roam-dailies-capture-date)
+(define-key org-roam-dailies-map (kbd ".") #'org-roam-dailies-find-directory)
 
 (provide 'org-roam-dailies)
 

org-roam-db.el

@@ -5,7 +5,7 @@
 ;; Author: Jethro Kuan <jethrokuan95@gmail.com>
 ;; URL: https://github.com/org-roam/org-roam
 ;; Keywords: org-mode, roam, convenience
-;; Version: 1.2.2
+;; Version: 1.2.3
 ;; Package-Requires: ((emacs "26.1") (dash "2.13") (f "0.17.2") (s "1.12.0") (org "9.3") (emacsql "3.0.0") (emacsql-sqlite3 "1.0.2"))
 
 ;; This file is NOT part of GNU Emacs.
@@ -35,17 +35,22 @@
 (require 'emacsql)
 (require 'emacsql-sqlite3)
 (require 'seq)
-(require 'org-macs)
+
-(require 'org-roam-macs)
+(eval-and-compile
+  (require 'org-roam-macs)
+  ;; For `org-with-wide-buffer'
+  (require 'org-macs))
 
 (defvar org-roam-directory)
 (defvar org-roam-enable-headline-linking)
 (defvar org-roam-verbose)
 (defvar org-roam-file-name)
 
+(defvar org-agenda-files)
+
 (declare-function org-roam--org-roam-file-p                "org-roam")
 (declare-function org-roam--extract-titles                 "org-roam")
-(declare-function org-roam--extract-ref                    "org-roam")
+(declare-function org-roam--extract-refs                   "org-roam")
 (declare-function org-roam--extract-tags                   "org-roam")
 (declare-function org-roam--extract-ids                    "org-roam")
 (declare-function org-roam--extract-links                  "org-roam")
@@ -79,11 +84,33 @@ value like `most-positive-fixnum'."
   :type 'int
   :group 'org-roam)
 
-(defconst org-roam-db--version 9)
+(defconst org-roam-db--version 10)
 
 (defvar org-roam-db--connection (make-hash-table :test #'equal)
   "Database connection to Org-roam database.")
 
+(defvar org-roam-db-dirty nil
+  "Whether the org-roam database is dirty and requires an update.
+Contains pairs of `org-roam-directory' and `org-roam-db-location'
+so that multi-directories are updated.")
+
+(defcustom org-roam-db-update-method 'idle-timer
+  "Method to update the Org-roam database.
+
+`immediate'
+  Update the database immediately upon file changes.
+
+`idle-timer'
+  Updates the database if dirty, if Emacs idles for `org-roam-db-update-idle-seconds'."
+  :type '(choice (const :tag "idle-timer" idle-timer)
+		 (const :tag "immediate" immediate))
+  :group 'org-roam)
+
+(defcustom org-roam-db-update-idle-seconds 2
+  "Number of idle seconds before triggering an Org-roam database update."
+  :type 'integer
+  :group 'org-roam)
+
 ;;;; Core Functions
 
 (defun org-roam-db--get-connection ()
@@ -107,7 +134,7 @@ Performs a database upgrade when required."
         (when init-db
           (org-roam-db--init conn))
         (let* ((version (caar (emacsql conn "PRAGMA user_version")))
-               (version (org-roam-db--update-maybe conn version)))
+               (version (org-roam-db--upgrade-maybe conn version)))
           (cond
            ((> version org-roam-db--version)
             (emacsql-close conn)
@@ -141,8 +168,8 @@ SQL can be either the emacsql vector representation, or a string."
       (level :not-null)])
 
     (links
-     [(from :not-null)
+     [(source :not-null)
-      (to :not-null)
+      (dest :not-null)
       (type :not-null)
       (properties :not-null)])
 
@@ -166,7 +193,7 @@ SQL can be either the emacsql vector representation, or a string."
       (emacsql db [:create-table $i1 $S2] table schema))
     (emacsql db (format "PRAGMA user_version = %s" org-roam-db--version))))
 
-(defun org-roam-db--update-maybe (db version)
+(defun org-roam-db--upgrade-maybe (db version)
   "Upgrades the database schema for DB, if VERSION is old."
   (emacsql-with-transaction db
     'ignore
@@ -191,6 +218,22 @@ the current `org-roam-directory'."
   (dolist (conn (hash-table-values org-roam-db--connection))
     (org-roam-db--close conn)))
 
+;;;; Timer-based updating
+(defvar org-roam-db-file-update-timer nil
+  "Timer for updating the database when dirty.")
+
+(defun org-roam-db-mark-dirty ()
+  "Mark the Org-roam database as dirty."
+  (add-to-list 'org-roam-db-dirty (list org-roam-directory org-roam-db-location)
+               nil #'equal))
+
+(defun org-roam-db-update-cache-on-timer ()
+  "Update the cache if the database is dirty.
+This function is called on `org-roam-db-file-update-timer'."
+  (pcase-dolist (`(,org-roam-directory ,org-roam-db-location) org-roam-db-dirty)
+    (org-roam-db-build-cache))
+  (setq org-roam-db-dirty nil))
+
 ;;;; Database API
 ;;;;; Initialization
 (defun org-roam-db--initialized-p ()
@@ -212,91 +255,152 @@ the current `org-roam-directory'."
     (dolist (table (mapcar #'car org-roam-db--table-schemata))
       (org-roam-db-query `[:delete :from ,table]))))
 
-(defun org-roam-db--clear-file (&optional filepath)
+(defun org-roam-db--clear-file (&optional file)
-  "Remove any related links to the file at FILEPATH.
+  "Remove any related links to the FILE.
 This is equivalent to removing the node from the graph."
-  (let ((file (expand-file-name (or filepath
+  (setq file (or file (buffer-file-name (buffer-base-buffer))))
-                                    (buffer-file-name (buffer-base-buffer))))))
+  (dolist (table (mapcar #'car org-roam-db--table-schemata))
-    (dolist (table (mapcar #'car org-roam-db--table-schemata))
+    (org-roam-db-query `[:delete :from ,table
-      (org-roam-db-query `[:delete :from ,table
+                         :where (= ,(if (eq table 'links) 'source 'file) $s1)]
-                           :where (= ,(if (eq table 'links) 'from 'file) $s1)]
+                       file)))
-                         file))))
 
-;;;;; Insertion
+;;;;; Inserting
-(defun org-roam-db--insert-meta (file hash meta)
+(defun org-roam-db--insert-meta (&optional update-p)
-  "Insert HASH and META for a FILE into the Org-roam cache."
+  "Update the metadata of the current buffer into the cache.
-  (org-roam-db-query
+If UPDATE-P is non-nil, first remove the meta for the file in the database."
-   [:insert :into files
+  (let* ((file (or org-roam-file-name (buffer-file-name)))
-    :values $v1]
+         (attr (file-attributes file))
-   (list (vector file hash meta))))
+         (atime (file-attribute-access-time attr))
+         (mtime (file-attribute-modification-time attr))
+         (hash (org-roam-db--file-hash)))
+    (when update-p
+      (org-roam-db-query [:delete :from files
+                          :where (= file $s1)]
+                         file))
+    (org-roam-db-query
+     [:insert :into files
+      :values $v1]
+     (list (vector file hash (list :atime atime :mtime mtime))))))
 
-(defun org-roam-db--insert-links (links)
+(defun org-roam-db--insert-titles (&optional update-p)
-  "Insert LINKS into the Org-roam cache."
+  "Update the titles of the current buffer into the cache.
-  (org-roam-db-query
+If UPDATE-P is non-nil, first remove titles for the file in the database.
-   [:insert :into links
+Returns the number of rows inserted."
-    :values $v1]
+  (let* ((file (or org-roam-file-name (buffer-file-name)))
-   links))
+         (titles (or (org-roam--extract-titles)
+                     (list (org-roam--path-to-slug file))))
+         (rows (mapcar (lambda (title)
+                         (vector file title)) titles)))
+    (when update-p
+      (org-roam-db-query [:delete :from titles
+                          :where (= file $s1)]
+                         file))
+    (org-roam-db-query
+        [:insert :into titles
+         :values $v1]
+        rows)
+    (length rows)))
 
-(defun org-roam-db--insert-titles (file titles)
+(defun org-roam-db--insert-refs (&optional update-p)
-  "Insert TITLES for a FILE into the Org-roam cache."
+  "Update the refs of the current buffer into the cache.
-  (org-roam-db-query
+If UPDATE-P is non-nil, first remove the ref for the file in the database."
-   [:insert :into titles
+  (let ((file (or org-roam-file-name (buffer-file-name)))
-    :values $v1]
+        (count 0))
-   (mapcar (lambda (title)
+    (when update-p
-             (vector file title)) titles)))
+      (org-roam-db-query [:delete :from refs
+                          :where (= file $s1)]
+                         file))
+    (when-let ((refs (org-roam--extract-refs)))
+      (dolist (ref refs)
+        (let ((key (cdr ref))
+              (type (car ref)))
+          (condition-case nil
+              (progn
+                (org-roam-db-query
+                 [:insert :into refs :values $v1]
+                 (list (vector key file type)))
+                (cl-incf count))
+            (error
+             (lwarn '(org-roam) :error
+                    (format "Duplicate ref %s in:\n\nA: %s\nB: %s\n\nskipping..."
+                            key
+                            file
+                            (caar (org-roam-db-query
+                                   [:select file :from refs
+                                    :where (= ref $v1)]
+                                   (vector key))))))))))
+    count))
 
-(defun org-roam-db--insert-ids (ids)
+(defun org-roam-db--insert-links (&optional update-p)
-  "Insert IDS into the Org-roam cache.
+  "Update the file links of the current buffer in the cache.
-Returns t if the insertion was successful, nil otherwise.
+If UPDATE-P is non-nil, first remove the links for the file in the database.
-Insertions can fail when there is an ID conflict."
+Return the number of rows inserted."
-  (condition-case nil
+  (let ((file (or org-roam-file-name (buffer-file-name))))
-      (progn
+    (when update-p
-        (org-roam-db-query
+      (org-roam-db-query [:delete :from links
-         [:insert :into ids
+                          :where (= source $s1)]
-          :values $v1]
+                         file))
-         ids)
+    (if-let ((links (org-roam--extract-links)))
-        t)
-    (error
-     (unless (listp ids)
-       (setq ids (list ids)))
-     (lwarn '(org-roam) :error
-            (format "Duplicate IDs in %s, one of:\n\n%s\n\nskipping..."
-                    (aref (car ids) 1)
-                    (string-join (mapcar (lambda (hl)
-                                           (aref hl 0)) ids) "\n")))
-     nil)))
-
-(defun org-roam-db--insert-tags (file tags)
-  "Insert TAGS for a FILE into the Org-roam cache."
-  (org-roam-db-query
-   [:insert :into tags
-    :values $v1]
-   (list (vector file tags))))
-
-(defun org-roam-db--insert-ref (file ref)
-  "Insert REF for FILE into the Org-roam cache.
-Returns t if successful, and nil otherwise.
-Insertions can fail if the key is already in the database."
-  (let ((key (cdr ref))
-        (type (car ref)))
-    (condition-case nil
         (progn
           (org-roam-db-query
-           [:insert :into refs :values $v1]
+           [:insert :into links
-           (list (vector key file type)))
+            :values $v1]
-          t)
+           links)
-      (error
+          (length links))
-       (lwarn '(org-roam) :error
+      0)))
-              (format "Duplicate ref %s in:\n\nA: %s\nB: %s\n\nskipping..."
+
-                      key
+(defun org-roam-db--insert-ids (&optional update-p)
-                      file
+  "Update the ids of the current buffer into the cache.
-                      (caar (org-roam-db-query
+If UPDATE-P is non-nil, first remove ids for the file in the database.
-                             [:select file :from refs
+Returns the number of rows inserted."
-                              :where (= ref $v1)]
+  (let ((file (or org-roam-file-name (buffer-file-name))))
-                             (vector key)))))
+    (when update-p
-       nil))))
+      (org-roam-db-query [:delete :from ids
+                          :where (= file $s1)]
+                         file))
+    (if-let ((ids (org-roam--extract-ids file)))
+      (condition-case nil
+          (progn
+            (org-roam-db-query
+             [:insert :into ids
+              :values $v1]
+             ids)
+            (length ids))
+        (error
+         (lwarn '(org-roam) :error
+                (format "Duplicate IDs in %s, one of:\n\n%s\n\nskipping..."
+                        (aref (car ids) 1)
+                        (string-join (mapcar (lambda (hl)
+                                               (aref hl 0)) ids) "\n")))
+         0))
+      0)))
+
+(defun org-roam-db--insert-tags (&optional update-p)
+  "Insert tags for the current buffer into the Org-roam cache.
+If UPDATE-P is non-nil, first remove tags for the file in the database.
+Return the number of rows inserted."
+  (let* ((file (or org-roam-file-name (buffer-file-name)))
+         (tags (org-roam--extract-tags file)))
+    (when update-p
+      (org-roam-db-query [:delete :from tags
+                          :where (= file $s1)]
+                         file))
+    (if tags
+        (progn (org-roam-db-query
+                [:insert :into tags
+                 :values $v1]
+                (list (vector file tags)))
+               1)
+      0)))
 
 ;;;;; Fetching
+(defun org-roam-db-has-file-p (file)
+  "Return t if FILE is in the database, nil otherwise."
+  (> (caar (org-roam-db-query [:select (funcall count) :from files
+                              :where (= file $s1)]
+                              file))
+     0))
+
 (defun org-roam-db--get-current-files ()
   "Return a hash-table of file to the hash of its file contents."
   (let* ((current-files (org-roam-db-query [:select * :from files]))
@@ -305,8 +409,8 @@ Insertions can fail if the key is already in the database."
       (puthash (car row) (cadr row) ht))
     ht))
 
-(defun org-roam-db--get-titles (file)
+(defun org-roam-db--get-title (file)
-  "Return the titles of FILE from the cache."
+  "Return the main title of FILE from the cache."
   (caar (org-roam-db-query [:select [title] :from titles
                             :where (= file $s1)
                             :limit 1]
@@ -329,12 +433,12 @@ If the file does not have any connections, nil is returned."
                    links_of(file, link) AS
                      (WITH filelinks AS (SELECT * FROM links WHERE NOT \"type\" = '\"cite\"'),
                            citelinks AS (SELECT * FROM links
-                                                  JOIN refs ON links.\"to\" = refs.\"ref\"
+                                                  JOIN refs ON links.\"dest\" = refs.\"ref\"
                                                             AND links.\"type\" = '\"cite\"')
-                      SELECT \"from\", \"to\" FROM filelinks UNION
+                      SELECT \"source\", \"dest\" FROM filelinks UNION
-                      SELECT \"to\", \"from\" FROM filelinks UNION
+                      SELECT \"dest\", \"source\" FROM filelinks UNION
-                      SELECT \"file\", \"from\" FROM citelinks UNION
+                      SELECT \"file\", \"source\" FROM citelinks UNION
-                      SELECT \"from\", \"file\" FROM citelinks),
+                      SELECT \"dest\", \"file\" FROM citelinks),
                    connected_component(file) AS
                      (SELECT link FROM links_of WHERE file = $s1
                       UNION
@@ -351,12 +455,12 @@ connections, nil is returned."
                    links_of(file, link) AS
                      (WITH filelinks AS (SELECT * FROM links WHERE NOT \"type\" = '\"cite\"'),
                            citelinks AS (SELECT * FROM links
-                                                  JOIN refs ON links.\"to\" = refs.\"ref\"
+                                                  JOIN refs ON links.\"dest\" = refs.\"ref\"
                                                             AND links.\"type\" = '\"cite\"')
-                      SELECT \"from\", \"to\" FROM filelinks UNION
+                      SELECT \"source\", \"dest\" FROM filelinks UNION
-                      SELECT \"to\", \"from\" FROM filelinks UNION
+                      SELECT \"dest\", \"source\" FROM filelinks UNION
-                      SELECT \"file\", \"from\" FROM citelinks UNION
+                      SELECT \"file\", \"source\" FROM citelinks UNION
-                      SELECT \"from\", \"file\" FROM citelinks),
+                      SELECT \"source\", \"file\" FROM citelinks),
                    -- Links are traversed in a breadth-first search.  In order to calculate the
                    -- distance of nodes and to avoid following cyclic links, the visited nodes
                    -- are tracked in 'trace'.
@@ -387,65 +491,6 @@ connections, nil is returned."
      (secure-hash 'sha1 (current-buffer)))))
 
 ;;;;; Updating
-(defun org-roam-db--update-meta ()
-  "Update the metadata of the current buffer into the cache."
-  (let* ((file (or org-roam-file-name (buffer-file-name)))
-         (attr (file-attributes file))
-         (atime (file-attribute-access-time attr))
-         (mtime (file-attribute-modification-time attr))
-         (hash (org-roam-db--file-hash)))
-    (org-roam-db-query [:delete :from files
-                        :where (= file $s1)]
-                       file)
-    (org-roam-db--insert-meta file hash (list :atime atime :mtime mtime))))
-
-(defun org-roam-db--update-titles ()
-  "Update the title of the current buffer into the cache."
-  (let* ((file (or org-roam-file-name (buffer-file-name)))
-         (titles (or (org-roam--extract-titles)
-                     (list (org-roam--path-to-slug file)))))
-    (org-roam-db-query [:delete :from titles
-                        :where (= file $s1)]
-                       file)
-    (org-roam-db--insert-titles file titles)))
-
-(defun org-roam-db--update-tags ()
-  "Update the tags of the current buffer into the cache."
-  (let* ((file (or org-roam-file-name (buffer-file-name)))
-         (tags (org-roam--extract-tags file)))
-    (org-roam-db-query [:delete :from tags
-                        :where (= file $s1)]
-                       file)
-    (when tags
-      (org-roam-db--insert-tags file tags))))
-
-(defun org-roam-db--update-refs ()
-  "Update the ref of the current buffer into the cache."
-  (let ((file (or org-roam-file-name (buffer-file-name))))
-    (org-roam-db-query [:delete :from refs
-                        :where (= file $s1)]
-                       file)
-    (when-let ((ref (org-roam--extract-ref)))
-      (org-roam-db--insert-ref file ref))))
-
-(defun org-roam-db--update-links ()
-  "Update the file links of the current buffer in the cache."
-  (let ((file (or org-roam-file-name (buffer-file-name))))
-    (org-roam-db-query [:delete :from links
-                        :where (= from $s1)]
-                       file)
-    (when-let ((links (org-roam--extract-links)))
-      (org-roam-db--insert-links links))))
-
-(defun org-roam-db--update-ids ()
-  "Update the ids of the current buffer into the cache."
-  (let* ((file (or org-roam-file-name (buffer-file-name))))
-    (org-roam-db-query [:delete :from ids
-                        :where (= file $s1)]
-                       file)
-    (when-let ((ids (org-roam--extract-ids file)))
-      (org-roam-db--insert-ids ids))))
-
 (defun org-roam-db--update-file (&optional file-path)
   "Update Org-roam cache for FILE-PATH.
 If the file does not exist anymore, remove it from the cache.
@@ -460,13 +505,13 @@ If the file exists, update the cache with information."
         (save-buffer)))
     (org-roam--with-temp-buffer file-path
       (emacsql-with-transaction (org-roam-db)
-        (org-roam-db--update-meta)
+        (org-roam-db--insert-meta 'update)
-        (org-roam-db--update-tags)
+        (org-roam-db--insert-tags 'update)
-        (org-roam-db--update-titles)
+        (org-roam-db--insert-titles 'update)
-        (org-roam-db--update-refs)
+        (org-roam-db--insert-refs 'update)
         (when org-roam-enable-headline-linking
-          (org-roam-db--update-ids))
+          (org-roam-db--insert-ids 'update))
-        (org-roam-db--update-links)))))
+        (org-roam-db--insert-links 'update)))))
 
 (defun org-roam-db-build-cache (&optional force)
   "Build the cache for `org-roam-directory'.
@@ -476,78 +521,112 @@ If FORCE, force a rebuild of the cache from scratch."
   (org-roam-db--close) ;; Force a reconnect
   (org-roam-db) ;; To initialize the database, no-op if already initialized
   (let* ((gc-cons-threshold org-roam-db-gc-threshold)
-	 (org-agenda-files nil)
+         (org-agenda-files nil)
          (org-roam-files (org-roam--list-all-files))
          (current-files (org-roam-db--get-current-files))
-         (file-count 0)
+         (count-plist nil)
+         (deleted-count 0)
+         (modified-files nil))
+    (dolist (file org-roam-files)
+      (let ((contents-hash (org-roam-db--file-hash file)))
+        (unless (string= (gethash file current-files)
+                       contents-hash)
+          (push (cons file contents-hash) modified-files)))
+      (remhash file current-files))
+    (dolist (file (hash-table-keys current-files))
+        ;; These files are no longer around, remove from cache...
+        (org-roam-db--clear-file file)
+        (setq deleted-count (1+ deleted-count)))
+    (setq count-plist (org-roam-db--update-files modified-files))
+    (org-roam-message "total: Δ%s, files-modified: Δ%s, ids: Δ%s, links: Δ%s, tags: Δ%s, titles: Δ%s, refs: Δ%s, deleted: Δ%s"
+                      (- (length org-roam-files) (plist-get count-plist :error-count))
+                      (plist-get count-plist :modified-count)
+                      (plist-get count-plist :id-count)
+                      (plist-get count-plist :link-count)
+                      (plist-get count-plist :tag-count)
+                      (plist-get count-plist :title-count)
+                      (plist-get count-plist :ref-count)
+                      deleted-count)))
+
+(defun org-roam-db--get-file-hash-from-db (&optional file-path)
+  "Get hash from Org-roam database for FILE-PATH."
+  (setq file-path (or file-path
+                      (buffer-file-name (buffer-base-buffer))))
+  (caar (org-roam-db-query [:select hash :from files
+                              :where (= file $s1)] file-path)))
+
+(defun org-roam-db-update-file (file-path)
+  "Update Org-roam cache for FILE-PATH.
+If the file does not exist anymore, remove it from the cache.
+If the file exists, update the cache with information."
+  (let ((content-hash (org-roam-db--file-hash file-path))
+        (db-hash  (org-roam-db--get-file-hash-from-db file-path)))
+    (unless (string= content-hash db-hash)
+      (org-roam-db--update-files (list (cons file-path content-hash)))
+      (org-roam-message "Updated: %s" file-path))))
+
+(defun org-roam-db--update-files (modified-files)
+  "Update Org-roam cache for a list of MODIFIED-FILES.
+FILES is a list of (file . hash) pairs."
+  (let* ((gc-cons-threshold org-roam-db-gc-threshold)
+         (org-agenda-files nil)
+         (error-count 0)
          (id-count 0)
          (link-count 0)
          (tag-count 0)
          (title-count 0)
          (ref-count 0)
-         (deleted-count 0)
+         (modified-count 0))
-         (processed-count 0))
+    (pcase-dolist (`(,file . _) modified-files)
-    (emacsql-with-transaction (org-roam-db)
+      (org-roam-db--clear-file file))
-      ;; Two-step building
+    ;; Process all the files for IDs first
-      ;; First step: Rebuild files and ids
+    ;;
-      (dolist (file org-roam-files)
+    ;; We do this so that link extraction is cheaper: this eliminates the need
-        (org-roam-message "Processed %s/%s files..." processed-count (length org-roam-files))
+    ;; to read the file to check if the ID really exists
-        (let* ((attr (file-attributes file))
+    (pcase-dolist (`(,file . ,contents-hash) modified-files)
-               (atime (file-attribute-access-time attr))
+      (let* ((attr (file-attributes file))
-               (mtime (file-attribute-modification-time attr)))
+             (atime (file-attribute-access-time attr))
-          (let ((contents-hash (org-roam-db--file-hash file)))
+             (mtime (file-attribute-modification-time attr)))
-            (unless (string= (gethash file current-files)
+        (condition-case nil
-                             contents-hash)
+            (org-roam--with-temp-buffer file
-              (condition-case nil
+              (org-roam-db-query
-                  (org-roam--with-temp-buffer file
+               [:insert :into files
-                    (org-roam-db--clear-file file)
+                :values $v1]
-                    (org-roam-db-query
+               (vector file contents-hash (list :atime atime :mtime mtime)))
-                     [:insert :into files
+              (when org-roam-enable-headline-linking
-                      :values $v1]
+                (setq id-count (+ id-count (org-roam-db--insert-ids)))))
-                     (vector file contents-hash (list :atime atime :mtime mtime)))
+          (file-error
-                    (setq file-count (1+ file-count))
+           (setq error-count (1+ error-count))
-                    (when org-roam-enable-headline-linking
+           (org-roam-db--clear-file file)
-                      (when-let ((ids (org-roam--extract-ids file)))
+           (lwarn '(org-roam) :warning
-                        (when (org-roam-db--insert-ids ids)
+                  "Skipping unreadable file while building cache: %s" file)))))
-                          (setq id-count (+ id-count (length ids))))))
+
-                    (when-let (links (org-roam--extract-links file))
+    ;; Process titles, tags, links and ref links of file
-                      (org-roam-db-query
+    (pcase-dolist (`(,file . _) modified-files)
-                       [:insert :into links
+      (org-roam-message "Processed %s/%s modified files..." modified-count (length modified-files))
-                        :values $v1]
+      (condition-case nil
-                       links)
+          (org-roam--with-temp-buffer file
-                      (setq link-count (1+ link-count)))
+            (setq modified-count (1+ modified-count))
-                    (when-let (tags (org-roam--extract-tags file))
+            (setq link-count (+ link-count (org-roam-db--insert-links)))
-                      (org-roam-db-query
+            (setq tag-count (+ tag-count (org-roam-db--insert-tags)))
-                       [:insert :into tags
+            (setq title-count (+ title-count (org-roam-db--insert-titles)))
-                        :values $v1]
+            (setq ref-count (+ ref-count (org-roam-db--insert-refs))))
-                       (vector file tags))
+        (file-error
-                      (setq tag-count (1+ tag-count)))
+         (setq error-count (1+ error-count))
-                    (let ((titles (or (org-roam--extract-titles)
+         (org-roam-db--clear-file file)
-                                      (list (org-roam--path-to-slug file)))))
+         (lwarn '(org-roam) :warning
-                      (org-roam-db--insert-titles file titles)
+                "Skipping unreadable file while building cache: %s" file))))
-                      (setq title-count (+ title-count (length titles))))
+    (list :error-count error-count :modified-count modified-count :id-count id-count :title-count title-count :tag-count tag-count :link-count link-count :ref-count ref-count)))
-                    (when-let* ((ref (org-roam--extract-ref)))
+
-                      (when (org-roam-db--insert-ref file ref)
+(defun org-roam-db-update ()
-                        (setq ref-count (1+ ref-count)))))
+  "Update the database."
-                (file-error
+  (pcase org-roam-db-update-method
-                 (setq org-roam-files (remove file org-roam-files))
+    ('immediate
-                 (org-roam-db--clear-file file)
+     (org-roam-db-update-file (buffer-file-name (buffer-base-buffer))))
-                 (lwarn '(org-roam) :warning
+    ('idle-timer
-                        "Skipping unreadable file while building cache: %s" file))))
+     (org-roam-db-mark-dirty))
-            (remhash file current-files)
+    (_
-            (setq processed-count (+ processed-count 1)))))
+     (user-error "Invalid `org-roam-db-update-method'"))))
-      (dolist (file (hash-table-keys current-files))
-        ;; These files are no longer around, remove from cache...
-        (org-roam-db--clear-file file)
-        (setq deleted-count (1+ deleted-count))))
-    (org-roam-message "files: Δ%s, ids: Δ%s, links: Δ%s, tags: Δ%s, titles: Δ%s, refs: Δ%s, deleted: Δ%s"
-                      file-count
-                      id-count
-                      link-count
-                      tag-count
-                      title-count
-                      ref-count
-                      deleted-count)))
 
 (provide 'org-roam-db)
 

org-roam-dev.el

@@ -5,7 +5,7 @@
 ;; Author: Jethro Kuan <jethrokuan95@gmail.com>
 ;; URL: https://github.com/org-roam/org-roam
 ;; Keywords: org-mode, roam, convenience
-;; Version: 1.2.2
+;; Version: 1.2.3
 ;; Package-Requires: ((emacs "26.1") (dash "2.13") (f "0.17.2") (s "1.12.0") (org "9.3") (emacsql "3.0.0") (emacsql-sqlite3 "1.0.2"))
 
 ;; This file is NOT part of GNU Emacs.

org-roam-doctor.el

@@ -5,7 +5,7 @@
 ;; Author: Jethro Kuan <jethrokuan95@gmail.com>
 ;; URL: https://github.com/jethrokuan/org-roam
 ;; Keywords: org-mode, roam, convenience
-;; Version: 1.2.2
+;; Version: 1.2.3
 ;; Package-Requires: ((emacs "26.1") (dash "2.13") (f "0.17.2") (s "1.12.0") (org "9.3") (emacsql "3.0.0") (emacsql-sqlite3 "1.0.2"))
 
 ;; This file is NOT part of GNU Emacs.
@@ -53,7 +53,6 @@
 (declare-function org-roam--get-roam-buffers "org-roam")
 (declare-function org-roam--list-all-files "org-roam")
 (declare-function org-roam--org-roam-file-p "org-roam")
-(declare-function org-roam--str-to-list "org-roam")
 (declare-function org-roam-mode "org-roam")
 
 (defvar org-roam-verbose)
@@ -122,7 +121,7 @@ AST is the org-element parse tree."
         (when (string-collate-equalp (org-element-property :key kw) "roam_tags" nil t)
           (let ((tags (org-element-property :value kw)))
             (condition-case nil
-                (org-roam--str-to-list tags)
+                (split-string-and-unquote tags)
               (error
                (push
                 `(,(org-element-property :begin kw)
@@ -142,7 +141,7 @@ AST is the org-element parse tree."
         (when (string-collate-equalp (org-element-property :key kw) "roam_alias" nil t)
           (let ((aliases (org-element-property :value kw)))
             (condition-case nil
-              (org-roam--str-to-list aliases)
+              (split-string-and-unquote aliases)
               (error
                (push
                 `(,(org-element-property :begin kw)

org-roam-faces.el

@@ -5,7 +5,7 @@
 ;; Author: Jethro Kuan <jethrokuan95@gmail.com>
 ;; URL: https://github.com/org-roam/org-roam
 ;; Keywords: org-mode, roam, convenience
-;; Version: 1.2.2
+;; Version: 1.2.3
 ;; Package-Requires: ((emacs "26.1"))
 
 ;; This file is NOT part of GNU Emacs.
@@ -65,6 +65,11 @@ This face is used on the region target by `org-roam-insertion'
 during an `org-roam-capture'."
   :group 'org-roam-faces)
 
+(defface org-roam-dailies-calendar-note
+  '((t :inherit (org-roam-link) :underline nil))
+  "Face for dates with a daily-note in the calendar"
+  :group 'org-roam-faces)
+
 ;;; _
 
 (provide 'org-roam-faces)

org-roam-graph.el

@@ -5,7 +5,7 @@
 ;; Author: Jethro Kuan <jethrokuan95@gmail.com>
 ;; URL: https://github.com/org-roam/org-roam
 ;; Keywords: org-mode, roam, convenience
-;; Version: 1.2.2
+;; Version: 1.2.3
 ;; Package-Requires: ((emacs "26.1") (dash "2.13") (f "0.17.2") (s "1.12.0") (org "9.3") (emacsql "3.0.0") (emacsql-sqlite3 "1.0.2"))
 
 ;; This file is NOT part of GNU Emacs.
@@ -32,7 +32,8 @@
 ;;; Code:
 (require 'xml) ;xml-escape-string
 (require 's)   ;s-truncate, s-replace
-(require 'org-roam-macs)
+(eval-and-compile
+  (require 'org-roam-macs))
 (require 'org-roam-db)
 
 ;;;; Declarations
@@ -169,15 +170,15 @@ into a digraph."
     (let* ((nodes (org-roam-db-query node-query))
            (edges-query
             `[:with selected :as [:select [file] :from ,node-query]
-              :select :distinct [to from] :from links
+              :select :distinct [dest source] :from links
-              :where (and (in to selected) (in from selected))])
+              :where (and (in dest selected) (in source selected))])
            (edges-cites-query
             `[:with selected :as [:select [file] :from ,node-query]
-              :select :distinct [file from]
+              :select :distinct [file source]
-              :from links :inner :join refs :on (and (= links:to refs:ref)
+              :from links :inner :join refs :on (and (= links:dest refs:ref)
                                                      (= links:type "cite")
                                                      (= refs:type "cite"))
-              :where (and (in file selected) (in from selected))])
+              :where (and (in file selected) (in source selected))])
            (edges       (org-roam-db-query edges-query))
            (edges-cites (org-roam-db-query edges-cites-query)))
       (insert "digraph \"org-roam\" {\n")

org-roam-link.el

@@ -6,7 +6,7 @@
 ;; Author: Jethro Kuan <jethrokuan95@gmail.com>
 ;; URL: https://github.com/org-roam/org-roam
 ;; Keywords: org-mode, roam, convenience
-;; Version: 1.2.2
+;; Version: 1.2.3
 ;; Package-Requires: ((emacs "26.1") (dash "2.13") (f "0.17.2") (s "1.12.0") (org "9.3") (emacsql "3.0.0") (emacsql-sqlite3 "1.0.2"))
 
 ;; This file is NOT part of GNU Emacs.
@@ -36,6 +36,10 @@
 
 (require 'ol)
 (require 'org-roam-compat)
+(require 'org-roam-macs)
+(require 'org-roam-db)
+
+(require 'org-element)
 
 (defvar org-roam-completion-ignore-case)
 (defvar org-roam-directory)
@@ -67,18 +71,21 @@ noabbrev  Absolute path, no abbreviation of home directory."
 (org-link-set-parameters "roam"
                          :follow #'org-roam-link-follow-link)
 
-(defun org-roam-link-follow-link (path)
+(defun org-roam-link-follow-link (_path)
-  "Navigates to location specified by PATH."
+  "Navigates to location in Org-roam link.
-  (pcase-let ((`(,link-type ,loc ,desc ,mkr) (org-roam-link--get-location path)))
+This function is called by Org when following links of the type
+`roam'. While the path is passed, assume that the cursor is on
+the link."
+  (pcase-let ((`(,link-type ,loc ,desc ,mkr) (org-roam-link--get-location)))
     (when (and org-roam-link-auto-replace loc desc)
       (org-roam-link--replace-link link-type loc desc))
     (pcase link-type
-          ("file"
+      ("file"
-           (if loc
+       (if loc
-               (org-roam--find-file loc)
+           (org-roam--find-file loc)
-             (org-roam-find-file desc nil nil t)))
+         (org-roam-find-file desc nil nil t)))
-          ("id"
+      ("id"
-           (org-goto-marker-or-bmk mkr)))))
+       (org-goto-marker-or-bmk mkr)))))
 
 ;;; Retrieval Functions
 (defun org-roam-link--get-titles ()
@@ -90,15 +97,11 @@ noabbrev  Absolute path, no abbreviation of home directory."
 If FILE, return outline headings for passed FILE instead.
 If WITH-MARKER, return a cons cell of (headline . marker).
 If USE-STACK, include the parent paths as well."
-  (let* ((buf (or (and file
+  (org-roam-with-file file (when with-marker 'keep)
-                       (or (find-buffer-visiting file)
+    (let* ((outline-level-fn outline-level)
-                           (find-file-noselect file)))
+           (path-separator "/")
-                  (current-buffer)))
+           (stack-level 0)
-         (outline-level-fn outline-level)
+           stack cands name level marker)
-         (path-separator "/")
-         (stack-level 0)
-         stack cands name level marker)
-    (with-current-buffer buf
       (save-excursion
         (goto-char (point-min))
         (while (re-search-forward org-complex-heading-regexp nil t)
@@ -122,8 +125,9 @@ If USE-STACK, include the parent paths as well."
                                     path-separator)))
             (push (if with-marker
                       (cons name marker)
-                    name) cands)))))
+                    name) cands))))
-    (nreverse cands)))
+      (nreverse cands))))
+
 
 (defun org-roam-link--get-file-from-title (title &optional no-interactive)
   "Return the file path corresponding to TITLE.
@@ -139,14 +143,11 @@ When NO-INTERACTIVE, return nil if there are multiple options."
          (completing-read "Select file: " files))))))
 
 (defun org-roam-link--get-id-from-headline (headline &optional file)
-  "Return (marker . id) correspondng to HEADLINE.
+  "Return (marker . id) correspondng to HEADLINE in FILE.
-If FILE, get headline from FILE instead.
+If FILE is nil, get ID from current buffer.
 If there is no corresponding headline, return nil."
   (save-excursion
-    (with-current-buffer (or (and file
+    (org-roam-with-file file 'keep
-                                  (or (find-buffer-visiting file)
-                                      (find-file-noselect file)))
-                             (current-buffer))
       (let ((headlines (org-roam-link--get-headlines file 'with-markers)))
         (when-let ((marker (cdr (assoc-string headline headlines))))
           (goto-char marker)
@@ -155,11 +156,11 @@ If there is no corresponding headline, return nil."
                   (org-id-get-create))))))))
 
 ;;; Path-related functions
-(defun org-roam-link-get-path (path)
+(defun org-roam-link-get-path (path &optional type)
   "Return the PATH of the link to use.
-Respect `org-link-file-path-type', see the variable documentation for details.
+If TYPE is non-nil, create a link of TYPE. Otherwise, respect
-If DIR is passed, use DIR as the default directory."
+`org-link-file-path-type'."
-  (pcase org-roam-link-file-path-type
+  (pcase (or type org-roam-link-file-path-type)
       ('absolute
        (abbreviate-file-name (expand-file-name path)))
       ('noabbrev
@@ -187,43 +188,59 @@ star-idx is the index of the asterisk, if any."
                        (t 'title+headline))))
       (list type title headline star-index))))
 
-(defun org-roam-link--get-location (link)
+(defun org-roam-link--get-location ()
-  "Return the location of Org-roam fuzzy LINK.
+  "Return the location of the Org-roam fuzzy link at point.
 The location is returned as a list containing (link-type loc desc marker).
 nil is returned if there is no matching location.
 
 link-type is either \"file\" or \"id\".
 loc is the target location: e.g. a file path, or an id.
-marker is a marker to the headline, if applicable."
+marker is a marker to the headline, if applicable.
-  (let (mkr link-type desc loc)
+
-    (pcase-let ((`(,type ,title ,headline _) (org-roam-link--split-path link)))
+desc is either the the description of the link under point, or
-      (pcase type
+the target of LINK (title or heading content)."
-        ('title+headline
+  (let ((context (org-element-context))
-         (let ((file (org-roam-link--get-file-from-title title)))
+        mkr link-type desc loc)
-           (if (not file)
+    (pcase (org-element-lineage context '(link) t)
-               (org-roam-message "Cannot find matching file")
+      (`nil (error "Not at an Org link"))
-             (setq mkr (org-roam-link--get-id-from-headline headline file))
+      (link
-             (pcase mkr
+       (if (not (string-equal "roam" (org-element-property :type link)))
-               (`(,marker . ,target-id)
+           (error "Not at Org-roam link")
-                (setq mkr marker
+         (setq desc (and (org-element-property :contents-begin link)
-                      loc target-id
+                         (org-element-property :contents-end link)
-                      link-type "id"
+                         (buffer-substring-no-properties
-                      desc headline))
+                          (org-element-property :contents-begin link)
-               (_ (org-roam-message "cannot find matching id"))))))
+                          (org-element-property :contents-end link))))
-        ('title
+         (pcase-let ((`(,type ,title ,headline _) (org-roam-link--split-path
-         (setq loc (org-roam-link--get-file-from-title title)
+                                                   (org-element-property :path link))))
-               desc title
+           (pcase type
-               link-type "file"))
+             ('title+headline
-        ('headline
+              (let ((file (org-roam-link--get-file-from-title title)))
-         (setq mkr (org-roam-link--get-id-from-headline headline))
+                (if (not file)
-         (pcase mkr
+                    (org-roam-message "Cannot find matching file")
-           (`(,marker . ,target-id)
+                  (setq mkr (org-roam-link--get-id-from-headline headline file))
-            (setq mkr marker
+                  (pcase mkr
-                  loc target-id
+                    (`(,marker . ,target-id)
-                  desc headline
+                     (progn
-                  link-type "id"))
+                       (setq mkr marker
-           (_ (org-roam-message "Cannot find matching headline")))))
+                             loc target-id
-      (list link-type loc desc mkr))))
+                             desc (or desc headline)
+                             link-type "id")))
+                    (_ (org-roam-message "Cannot find matching id"))))))
+             ('title
+              (setq loc (org-roam-link--get-file-from-title title)
+                    link-type "file"
+                    desc (or desc title)))
+             ('headline
+              (setq mkr (org-roam-link--get-id-from-headline headline))
+              (pcase mkr
+                (`(,marker . ,target-id)
+                 (setq mkr marker
+                       loc target-id
+                       link-type "id"
+                       desc (or desc headline)))
+                (_ (org-roam-message "Cannot find matching headline")))))))))
+    (list link-type loc desc mkr)))
 
 ;;; Conversion Functions
 (defun org-roam-link--replace-link (link-type loc &optional desc)
@@ -244,14 +261,11 @@ DESC is the link description."
   (save-excursion
     (goto-char (point-min))
     (while (re-search-forward org-link-bracket-re nil t)
-      (let ((context (org-element-context)))
+      (condition-case nil
-          (pcase (org-element-lineage context '(link) t)
+          (pcase-let ((`(,link-type ,loc ,desc _) (org-roam-link--get-location)))
-            (`nil nil)
+            (when (and link-type loc)
-            (link
+              (org-roam-link--replace-link link-type loc desc)))
-             (when (string-equal "roam" (org-element-property :type link))
+        (error nil)))))
-               (pcase-let ((`(,link-type ,loc ,desc _) (org-roam-link--get-location (org-element-property :path link))))
-                 (when (and link-type loc)
-                   (org-roam-link--replace-link link-type loc desc))))))))))
 
 (defun org-roam-link--replace-link-on-save ()
   "Hook to replace all roam links on save."
@@ -304,7 +318,7 @@ DESC is the link description."
                                 (if headline-only-p 1 0)))
                 (insert (concat (unless (string= link-type "roam") "roam:")
                                 (when headline-only-p "*")
-                                str))))))))
+                                (org-link-escape str)))))))))
 
 (provide 'org-roam-link)
 ;;; org-roam-link.el ends here

org-roam-macs.el

@@ -5,7 +5,7 @@
 ;; Author: Jethro Kuan <jethrokuan95@gmail.com>
 ;; URL: https://github.com/org-roam/org-roam
 ;; Keywords: org-mode, roam, convenience
-;; Version: 1.2.2
+;; Version: 1.2.3
 ;; Package-Requires: ((emacs "26.1") (dash "2.13") (f "0.17.2") (s "1.12.0") (org "9.3") (emacsql "3.0.0") (emacsql-sqlite3 "1.0.2"))
 
 ;; This file is NOT part of GNU Emacs.
@@ -34,9 +34,15 @@
 ;;; Code:
 ;;;; Library Requires
 (require 'dash)
+(require 's)
 
 (defvar org-roam-verbose)
 
+;; This is necessary to ensure all dependents on this module see
+;; `org-mode-hook' and `org-inhibit-startup' as dynamic variables,
+;; regardless of whether Org is loaded before their compilation.
+(require 'org)
+
 ;;;; Utility Functions
 (defun org-roam--list-interleave (lst separator)
   "Interleaves elements in LST with SEPARATOR."
@@ -46,6 +52,28 @@
         (nconc new-lst (list separator it)))
       new-lst)))
 
+(defmacro org-roam-with-file (file keep-buf-p &rest body)
+  "Execute BODY within FILE.
+If FILE is nil, execute BODY in the current buffer.
+Kills the buffer if KEEP-BUF-P is nil, and FILE is not yet visited."
+  (declare (indent 2) (debug t))
+  `(let* (new-buf
+          (buf (or (and (not ,file)
+                        (current-buffer)) ;If FILE is nil, use current buffer
+                   (find-buffer-visiting ,file) ; If FILE is already visited, find buffer
+                   (progn
+                     (setq new-buf t)
+                     (find-file-noselect ,file)))) ; Else, visit FILE and return buffer
+          res)
+     (with-current-buffer buf
+       (setq res (progn ,@body))
+       (unless (and new-buf (not ,keep-buf-p))
+         (save-buffer)))
+     (if (and new-buf (not ,keep-buf-p))
+       (when (find-buffer-visiting ,file)
+         (kill-buffer (find-buffer-visiting ,file))))
+     res))
+
 (defmacro org-roam--with-temp-buffer (file &rest body)
   "Execute BODY within a temp buffer.
 Like `with-temp-buffer', but propagates `org-roam-directory'.
@@ -60,7 +88,8 @@ If FILE, set `org-roam-temp-file-name' to file and insert its contents."
            (org-mode)
            (when ,file
              (insert-file-contents ,file)
-             (setq-local org-roam-file-name ,file))
+             (setq-local org-roam-file-name ,file)
+             (setq-local default-directory (file-name-directory ,file)))
            ,@body)))))
 
 (defun org-roam-message (format-string &rest args)

org-roam-protocol.el

@@ -4,7 +4,7 @@
 ;; Author: Jethro Kuan <jethrokuan95@gmail.com>
 ;; URL: https://github.com/org-roam/org-roam
 ;; Keywords: org-mode, roam, convenience
-;; Version: 1.2.2
+;; Version: 1.2.3
 ;; Package-Requires: ((emacs "26.1") (org "9.3"))
 
 ;; This file is NOT part of GNU Emacs.
@@ -39,6 +39,11 @@
 (require 'org-roam)
 (require 'ol) ;; for org-link-decode
 
+(defcustom org-roam-protocol-store-links nil
+  "Whether to store links when capturing websites with `org-roam-protocol'."
+  :type 'boolean
+  :group 'org-roam)
+
 ;;;; Functions
 (defun org-roam-protocol-open-ref (info)
   "Process an org-protocol://roam-ref?ref= style url with INFO.
@@ -46,7 +51,7 @@
 It opens or creates a note with the given ref.
 
   javascript:location.href = \\='org-protocol://roam-ref?template=r&ref=\\='+ \\
-        encodeURIComponent(location.href) + \\='&title=\\=' \\
+        encodeURIComponent(location.href) + \\='&title=\\=' + \\
         encodeURIComponent(document.title) + \\='&body=\\=' + \\
         encodeURIComponent(window.getSelection())"
   (when-let* ((alist (org-roam--plist-to-alist info))
@@ -58,9 +63,24 @@ It opens or creates a note with the given ref.
       (error "No ref key provided"))
     (when-let ((title (cdr (assoc 'title decoded-alist))))
       (push (cons 'slug (funcall org-roam-title-to-slug-function title)) decoded-alist))
+    (let-alist decoded-alist
+      (let* ((ref (org-protocol-sanitize-uri .ref))
+             (type (and (string-match org-link-plain-re ref)
+                        (match-string 1 ref)))
+             (title (or .title ""))
+             (body (or .body ""))
+             (orglink
+              (org-link-make-string ref (or (org-string-nw-p title) ref))))
+        (when org-roam-protocol-store-links
+          (push (list ref title) org-stored-links))
+        (org-link-store-props :type type
+                              :link ref
+                              :annotation orglink
+                              :initial body)))
     (let* ((org-roam-capture-templates org-roam-capture-ref-templates)
            (org-roam-capture--context 'ref)
            (org-roam-capture--info decoded-alist)
+           (org-capture-link-is-already-stored t)
            (template (cdr (assoc 'template decoded-alist))))
       (raise-frame)
       (org-roam-capture--capture nil template)

tests/roam-files/cite_ref.org

@@ -0,0 +1 @@
+#+roam_key: cite:mitsuha2007

tests/roam-files/multiple-refs.org

@@ -0,0 +1,2 @@
+#+roam_key: https://www.orgroam.com/
+#+roam_key: cite:orgroam2020

tests/test-org-roam-perf.el

@@ -45,7 +45,7 @@
     (pcase (benchmark-run 1 (org-roam-db-build-cache t))
       (`(,time ,gcs ,time-in-gc)
        (message "Elapsed time: %fs (%fs in %d GCs)" time time-in-gc gcs)
-       (expect time :to-be-less-than 90))))
+       (expect time :to-be-less-than 110))))
   (it "builds quickly without change"
     (pcase (benchmark-run 1 (org-roam-db-build-cache))
       (`(,time ,gcs ,time-in-gc)

tests/test-org-roam.el

@@ -52,24 +52,39 @@
   (delete-file org-roam-db-location)
   (org-roam-db--close))
 
-(describe "org-roam--str-to-list"
+(describe "Ref extraction"
-  (it "nil"
+  (before-all
-    (expect (org-roam--str-to-list nil)
+    (test-org-roam--init))
-            :to-be
+
-            nil))
+  (after-all
-  (it "\"multi word\" prop 123"
+    (test-org-roam--teardown))
-    (expect (org-roam--str-to-list "\"multi word\" prop 123")
+
-            :to-equal
+  (cl-flet
-            '("multi word" "prop" "123")))
+      ((test (fn file)
-  (it "prop \"multi word\" 123"
+             (let* ((fname (test-org-roam--abs-path file))
-    (expect (org-roam--str-to-list "\"multi word\" prop 123")
+                    (buf (find-file-noselect fname)))
-            :to-equal
+               (with-current-buffer buf
-            '("multi word" "prop" "123")))
+                 ;; Unlike tag extraction, it doesn't make sense to
-  (it "errors on bad input"
+                 ;; pass a filename.
-    (expect (org-roam--str-to-list 1)
+                 (funcall fn)))))
-            :to-throw)
+    ;; Enable "cite:" link parsing
-    (expect (org-roam--str-to-list "\"hello")
+    (org-link-set-parameters "cite")
-            :to-throw)))
+    (it "extracts web keys"
+      (expect (test #'org-roam--extract-ref
+                    "web_ref.org")
+              :to-equal
+              '("website" . "//google.com/")))
+    (it "extracts cite keys"
+      (expect (test #'org-roam--extract-ref
+                    "cite_ref.org")
+              :to-equal
+              '("cite" . "mitsuha2007")))
+    (it "extracts all keys"
+      (expect (test #'org-roam--extract-refs
+                    "multiple-refs.org")
+              :to-have-same-items-as
+              '(("cite" . "orgroam2020")
+                ("website" . "//www.orgroam.com/"))))))
 
 (describe "Title extraction"
   :var (org-roam-title-sources)
@@ -300,21 +315,21 @@
 
     ;; Links
     (expect (caar (org-roam-db-query [:select (funcall count) :from links
-                                      :where (= from $s1)]
+                                      :where (= source $s1)]
                                      (test-org-roam--abs-path "foo.org"))) :to-be 1)
     (expect (caar (org-roam-db-query [:select (funcall count) :from links
-                                      :where (= from $s1)]
+                                      :where (= source $s1)]
                                      (test-org-roam--abs-path "nested/bar.org"))) :to-be 2)
 
     ;; Links -- File-to
     (expect (caar (org-roam-db-query [:select (funcall count) :from links
-                                      :where (= to $s1)]
+                                      :where (= dest $s1)]
                                      (test-org-roam--abs-path "nested/foo.org"))) :to-be 1)
     (expect (caar (org-roam-db-query [:select (funcall count) :from links
-                                      :where (= to $s1)]
+                                      :where (= dest $s1)]
                                      (test-org-roam--abs-path "nested/bar.org"))) :to-be 1)
     (expect (caar (org-roam-db-query [:select (funcall count) :from links
-                                      :where (= to $s1)]
+                                      :where (= dest $s1)]
                                      (test-org-roam--abs-path "unlinked.org"))) :to-be 0)
     ;; TODO Test titles
     (expect (org-roam-db-query [:select * :from titles])