(release): v1.1.0-rc1 (#238)

This lets the viewer application watch the file, or have the user manually refresh the view rather than opening a new browser tab.

.github/CONTRIBUTING.md

@@ -0,0 +1,34 @@
+# Contributing
+
+If you discover issues, have ideas for improvements or new features, please
+report them to the [issue tracker][1] of the repository or submit a pull
+request. Please, try to follow these guidelines when you do so.
+
+## Issue reporting
+
+* Check that the issue has not already been reported.
+* Check that the issue has not already been fixed in the latest code
+  (a.k.a. `develop`).
+* Be clear, concise and precise in your description of the problem.
+* Open an issue with a descriptive title and a summary in grammatically correct,
+  complete sentences.
+* Include any relevant code to the issue summary.
+* If you're reporting performance issues it'd be nice if you added some profiling data (Emacs has a built-in profiler).
+
+## Pull requests
+
+* Read [how to properly contribute to open source projects on Github][2].
+* Use a topic branch to easily amend a pull request later, if necessary.
+* Write [good commit messages][3].
+* Mention related tickets in the commit messages (e.g. `[Fix #N] Add missing autoload cookies`)
+* Update the [changelog][5].
+* Use the same coding conventions as the rest of the project.
+* Verify your Emacs Lisp code with `checkdoc` (<kbd>C-c ? d</kbd>).
+* Open a [pull request][4] that relates to *only* one subject with a clear title
+  and description in grammatically correct, complete sentences.
+
+[1]: https://github.com/jethrokuan/org-roam/issues
+[2]: http://gun.io/blog/how-to-github-fork-branch-and-pull-request
+[3]: http://tbaggery.com/2008/04/19/a-note-about-git-commit-messages.html
+[4]: https://help.github.com/articles/using-pull-requests
+[5]: https://github.com/jethrokuan/org-roam/blob/master/CHANGELOG.md

.github/ISSUE_TEMPLATE/bug_report.md

@@ -0,0 +1,30 @@
+---
+name: Bug Report
+about: Something's not working.
+title: ''
+labels: ''
+assignees: 'jethrokuan'
+---
+
+### Description
+
+#### Steps to Reproduce
+<!--
+Example:
+
+1. Load Emacs
+2. Run `org-roam--build-cache-async`
+3. Run `org-roam-find-file`
+...
+-->
+
+
+#### Expected Results
+<!-- Example: File A is there -->
+
+#### Actual Results
+<!-- Example: File A is missing -->
+
+### Versions
+- Emacs (`C-h v emacs-version`): vX.X.X
+- Org-roam commit: https://github.com/jethrokuan/org-roam/commit/commithashhere

.github/ISSUE_TEMPLATE/feature_request.md

@@ -0,0 +1,16 @@
+---
+name: Feature Request
+about: Create a feature request to improve Org-roam
+title: ''
+labels: 'enhancement'
+assignees: 'jethrokuan'
+---
+
+### Brief Abstract
+
+### Long Description
+
+### Proposed Implementation (if any)
+
+### Please check the following:
+- [ ] No similar feature requests

.github/PULL_REQUEST_TEMPLATE.md

@@ -0,0 +1 @@
+###### Motivation for this change

.github/workflows/test.yml

@@ -0,0 +1,69 @@
+# * test.yml --- Test Emacs packages using makem.sh on GitHub Actions
+
+# https://github.com/alphapapa/makem.sh
+
+# Based on Steve Purcell's examples at
+# <https://github.com/purcell/setup-emacs/blob/master/.github/workflows/test.yml>,
+# <https://github.com/purcell/package-lint/blob/master/.github/workflows/test.yml>.
+
+# * License:
+
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <https://www.gnu.org/licenses/>.
+
+# * Code:
+
+name: "CI"
+on:
+  pull_request:
+  push:
+    branches:
+      - master
+      - develop
+
+jobs:
+  build:
+    runs-on: ubuntu-latest
+    strategy:
+      fail-fast: false
+      matrix:
+        emacs_version:
+          - 26.3
+          - snapshot
+    steps:
+    - uses: purcell/setup-emacs@master
+      with:
+        version: ${{ matrix.emacs_version }}
+
+    - uses: actions/checkout@v2
+
+    - name: Initialize sandbox
+      run: |
+        SANDBOX_DIR=$(mktemp -d) || exit 1
+        echo ::set-env name=SANDBOX_DIR::$SANDBOX_DIR
+        ./makem.sh -vv --sandbox $SANDBOX_DIR --install-deps --install-linters
+
+    # The "all" rule is not used, because it treats compilation warnings
+    # as failures, so linting and testing are run as separate steps.
+
+    - name: Lint
+      continue-on-error: true
+      run: ./makem.sh -vv --sandbox $SANDBOX_DIR lint
+
+    - name: Test
+      if: always()  # Run test even if linting fails.
+      run: ./makem.sh -vv --sandbox $SANDBOX_DIR test
+
+# Local Variables:
+# eval: (outline-minor-mode)
+# End:

.gitignore

@@ -0,0 +1,2 @@
+/.sandbox/
+**/*.elc

CHANGELOG.md

@@ -0,0 +1,125 @@
+# Changelog
+
+## 1.0.0-rc1 (06-03-2020)
+
+This is a pre-release before the push to MELPA. It contains large
+internal changes, with little user-facing changes. Most notably, the
+backing storage has been changed to a SQLite database, and a
+templating system using `org-capture` is introduced.
+
+### Breaking Changes
+* [#200][gh-200] Move Org-roam cache into a SQLite database.
+* [#203][gh-203] Roam protocol is deprecated, in favour of extending org-roam-protocol.
+
+### New Features
+* [#182][gh-182] Support file name aliases via `#+ROAM_ALIAS`.
+* [#216][gh-216] Adds templating functionality by extending org-capture.
+* [#232][gh-232] Adds a prefix key to `org-roam-show-graph`, to generate graph without opening it.
+* [#233][gh-233] Adds `org-roam-graph-exclude-matcher`, which allows exclusion of nodes from graph.
+
+### Bugfixes
+* [#207][gh-207], [#221][gh-221] small bugfixes to Org-roam graph generation
+* [#230][gh-230] remove nonspacing marks from filenames, to prevent cross-platform errors
+
+### New Contributors
+* [@acowley][https://github.com/acowley]
+* [@teesloane][https://github.com/teesloane]
+
+## 0.1.2 (2020-02-21)
+
+### Breaking Changes
+* [#143][gh-143] `org-roam-mode` is now a global mode. The installation instructions have changed accordingly.
+* [#103][gh-103] Change `org-roam-file-format` to a function: `org-roam-file-name-function` to allow more flexible file name customizaton. Also changes `org-roam-use-timestamp-as-filename` to `org-roam-filename-noconfirm` to better describe what it does.
+
+### New Features
+* [#145][gh-145] `org-roam-show-graph`: Fallback to Emacs SVG viewer
+* [#141][gh-141] add variable `org-roam-new-file-directory` for new Org-roam files
+* [#138][gh-138] add `org-roam-switch-to-buffer`
+* [#124][gh-124], [#141][gh-141] Maintain cache consistency on file rename and delete
+* [#87][gh-87], [#90][gh-90] Support encrypted Org files
+* [#110][gh-110] Add prefix to `org-roam-insert`, for inserting titles down-cased
+* [#99][gh-99] Add keybinding so that `<return>` or `mouse-1` in the backlinks buffer visits the source file of the backlink at point
+
+### Changes
+
+* [#108][gh-108] Locally overwrite the link following behaviour in the org-roam-buffer to open files in the same window `org-roam` was called from
+
+### Bugfixes
+* [#86][gh-86] Fix `org-roam--parse-content` incorrect `:to` computation for nested files
+* [#98][gh-98] Fix `org-roam--find-file` picking up temporary files
+* [#136][gh-136] Misc bugfixes
+
+### Internal
+* [#122][gh-122], [#128][gh-128] Improve performance of post-command-hook
+* [#92][gh-92], [#105][gh-105] Add tests for core functionality
+
+### New Contributors
+* [@frigge](https://github.com/frigge)
+* [@juergenhoetzel](https://github.com/juergenhoetzel)
+* [@chip2n](https://github.com/chip2n)
+* [@l3kn](https://github.com/l3kn)
+* [@jdormit](https://github.com/jdormit)
+* [@herbertjones](https://github.com/herbertjones)
+* [@CeleritasCelery](https://github.com/CeleritasCelery)
+* [@daniel-koudouna](https://github.com/daniel-koudouna)
+
+## 0.1.1 (2020-02-15)
+
+Mostly a documentation/cleanup release.
+
+### New Features
+* [#62][gh-62] Add the options `org-roam-use-timestamps-as-filename` and `org-roam-file-format`, more in documentation.
+
+### Breaking Changes
+* [#62][gh-62] The ID (file-name) workflow is no longer first-class, but a fallback when titles don't exist.
+
+### Changes
+* [#66][gh-66], [#68][gh-68]: Improved the quality of the package in preparation of submission to MELPA
+* [#73][gh-73]: Added CI to the project via Github Issues (Thanks [@alphapapa](https://github.com/alphapapa/) for scripts and setup)
+* [#69][gh-69], [#72][gh-72], [#75][gh-75]: Major cleanup and de-duplication of code
+
+### Bugfixes
+* [#67][gh-67]: Fixed `org-roam--make-file` not creating files with extensions
+* [#71][gh-71], [#78][gh-78]: Fixed `org-roam-insert` not inserting correct paths
+* [#82][gh-82]: Fixed nested Org-roam files not being detected as part of Org-roam
+
+[gh-62]: https://github.com/jethrokuan/org-roam/pull/66
+[gh-66]: https://github.com/jethrokuan/org-roam/pull/66
+[gh-67]: https://github.com/jethrokuan/org-roam/pull/67
+[gh-68]: https://github.com/jethrokuan/org-roam/pull/68
+[gh-69]: https://github.com/jethrokuan/org-roam/pull/69
+[gh-71]: https://github.com/jethrokuan/org-roam/pull/71
+[gh-72]: https://github.com/jethrokuan/org-roam/pull/72
+[gh-73]: https://github.com/jethrokuan/org-roam/pull/73
+[gh-75]: https://github.com/jethrokuan/org-roam/pull/75
+[gh-78]: https://github.com/jethrokuan/org-roam/pull/78
+[gh-82]: https://github.com/jethrokuan/org-roam/pull/82
+[gh-86]: https://github.com/jethrokuan/org-roam/pull/86
+[gh-87]: https://github.com/jethrokuan/org-roam/pull/87
+[gh-90]: https://github.com/jethrokuan/org-roam/pull/90
+[gh-92]: https://github.com/jethrokuan/org-roam/pull/92
+[gh-98]: https://github.com/jethrokuan/org-roam/pull/98
+[gh-99]: https://github.com/jethrokuan/org-roam/pull/99
+[gh-103]: https://github.com/jethrokuan/org-roam/pull/103
+[gh-105]: https://github.com/jethrokuan/org-roam/pull/105
+[gh-108]: https://github.com/jethrokuan/org-roam/pull/108
+[gh-110]: https://github.com/jethrokuan/org-roam/pull/110
+[gh-122]: https://github.com/jethrokuan/org-roam/pull/122
+[gh-124]: https://github.com/jethrokuan/org-roam/pull/124
+[gh-128]: https://github.com/jethrokuan/org-roam/pull/128
+[gh-136]: https://github.com/jethrokuan/org-roam/pull/136
+[gh-138]: https://github.com/jethrokuan/org-roam/pull/138
+[gh-141]: https://github.com/jethrokuan/org-roam/pull/141
+[gh-142]: https://github.com/jethrokuan/org-roam/pull/142
+[gh-143]: https://github.com/jethrokuan/org-roam/pull/143
+[gh-182]: https://github.com/jethrokuan/org-roam/pull/182
+[gh-188]: https://github.com/jethrokuan/org-roam/pull/188
+[gh-200]: https://github.com/jethrokuan/org-roam/pull/200
+[gh-207]: https://github.com/jethrokuan/org-roam/pull/207
+[gh-216]: https://github.com/jethrokuan/org-roam/pull/216
+[gh-221]: https://github.com/jethrokuan/org-roam/pull/221
+[gh-230]: https://github.com/jethrokuan/org-roam/pull/230
+
+ # Local Variables:
+ # eval: (auto-fill-mode -1)
+ # End:

Makefile

@@ -0,0 +1,56 @@
+# * makem.sh/Makefile --- Script to aid building and testing Emacs Lisp packages
+
+# This Makefile is from the makem.sh repo: <https://github.com/alphapapa/makem.sh>.
+
+# * Arguments
+
+# For consistency, we use only var=val options, not hyphen-prefixed options.
+
+# NOTE: I don't like duplicating the arguments here and in makem.sh,
+# but I haven't been able to find a way to pass arguments which
+# conflict with Make's own arguments through Make to the script.
+# Using -- doesn't seem to do it.
+
+ifdef install-deps
+	INSTALL_DEPS = "--install-deps"
+endif
+ifdef install-linters
+	INSTALL_LINTERS = "--install-linters"
+endif
+
+ifdef sandbox
+	ifeq ($(sandbox), t)
+		SANDBOX = --sandbox
+	else
+		SANDBOX = --sandbox $(sandbox)
+	endif
+endif
+
+ifdef debug
+	DEBUG = "--debug"
+endif
+
+# ** Verbosity
+
+# Since the "-v" in "make -v" gets intercepted by Make itself, we have
+# to use a variable.
+
+verbose = $(v)
+
+ifneq (,$(findstring vv,$(verbose)))
+	VERBOSE = "-vv"
+else ifneq (,$(findstring v,$(verbose)))
+	VERBOSE = "-v"
+endif
+
+# * Rules
+
+# TODO: Handle cases in which "test" or "tests" are called and a
+# directory by that name exists, which can confuse Make.
+
+%:
+	@./makem.sh $(DEBUG) $(VERBOSE) $(SANDBOX) $(INSTALL_DEPS) $(INSTALL_LINTERS) $(@)
+
+.DEFAULT: init
+init:
+	@./makem.sh $(DEBUG) $(VERBOSE) $(SANDBOX) $(INSTALL_DEPS) $(INSTALL_LINTERS)

README.md

@@ -1,28 +1,26 @@
+[![License GPL 3][badge-license]](http://www.gnu.org/licenses/gpl-3.0.txt)
 [![Documentation Status](https://readthedocs.org/projects/org-roam/badge/?version=latest)](https://org-roam.readthedocs.io/en/latest/?badge=latest)
+[![GitHub Release](https://img.shields.io/github/v/release/jethrokuan/org-roam)](https://img.shields.io/github/v/release/jethrokuan/org-roam)
+
+## Synopsis
 
 Org-roam is a rudimentary [Roam][roamresearch] replica built around
 the all-powerful [Org-mode][org]. 
 
 Like Roam, Org-roam offers a powerful and effortless non-hierarchical
 note-taking approach. With Org-roam, notes flow naturally, making
-note-taking fun and easy. 
+note-taking fun and easy. Org-roam *enables* a note-taking workflow that
+is not fluid with vanilla Org-mode (more in [this blog
+post](https://blog.jethro.dev/posts/how_to_take_smart_notes_org/)).
+
 
 The goal of the project is to implement core features of Roam around
 Org-mode, and eventually introduce features enabled by the Emacs
-ecosystem. 
+ecosystem.
 
-For more documentation, see [the documentation page](https://org-roam.readthedocs.io/en/latest/).
-
-## Understanding Roam
-
-To understand more about Roam, I recommend the following links:
-
--   [Building a second brain in
-    Roam](https://reddit.com/r/RoamResearch/comments/eho7de/building_a_second_brain_in_roamand_why_you_might)
--   [Roam: Why I Love It and How I Use
-    It](https://www.nateliason.com/blog/roam)
-
-## Project Status
+Visit [the documentation
+page](https://org-roam.readthedocs.io/en/latest/) for a tutorial and
+more links.
 
 As of February 2020, it is in a very early stage of development. 
 
@@ -30,23 +28,55 @@ As of February 2020, it is in a very early stage of development.
 
 Here's a screenshot of `org-roam`. The `org-roam` buffer shows
 backlinks for the active org buffer in the left window, as well as the
-surrounding content in the backlink file. The backlink database is
-built asynchronously in the background, and is not noticeable to the
-end user. The graph is generated from the link structure, and can be
-used to navigate to the respective files.
+surrounding content in the backlink file. The database is built once,
+and updated incrementally. The graph is generated from the link
+structure, and can be used to navigate to the respective files.
 
 ![img](doc/images/org-roam-graph.gif)
 
-## Knowledge Bases using Org-Roam
+## Installation
+
+The recommended method is using use-package and straight, or a similar package manager.
+
+```emacs-lisp
+(use-package org-roam
+      :hook 
+      (after-init . org-roam-mode)
+      :straight (:host github :repo "jethrokuan/org-roam" :branch "develop")
+      :custom
+      (org-roam-directory "/path/to/org-files/")
+      :bind (:map org-roam-mode-map
+              (("C-c n l" . org-roam)
+               ("C-c n f" . org-roam-find-file)
+               ("C-c n g" . org-roam-show-graph))
+              :map org-mode-map
+              (("C-c n i" . org-roam-insert))))
+```
+
+For more detailed installation instructions (including instructions for
+Spacemacs users), please see [the installation
+documentation](https://org-roam.readthedocs.io/en/develop/installation/).
+
+## Knowledge Bases using Org-roam
 
 - [Jethro Kuan](https://braindump.jethro.dev/)
   ([Source](https://github.com/jethrokuan/braindump/tree/master/org))
 
+## Changelog
+
+A changelog is being maintained [here](CHANGELOG.md)
+
 ## Contributing
 
 To report bugs and suggest new feature use the issue tracker. If you
 have some code which you would like to be merged, then open a pull
-request. Please also see CONTRIBUTING.md.
+request. Please also see [CONTRIBUTING.md](.github/CONTRIBUTING.md).
+
+## License
+
+Copyright © Jethro Kuan and contributors. Distributed under the GNU
+General Public License, Version 3
 
 [roamresearch]: https://www.roamresearch.com/
 [org]: https://orgmode.org/
+[badge-license]: https://img.shields.io/badge/license-GPL_3-green.svg

doc/anatomy.md

@@ -0,0 +1,40 @@
+The bulk of Org-roam's functionality is built on top of vanilla
+Org-mode. However, to support additional functionality, Org-roam adds
+several Org-roam-specific keywords. These functionality are not
+crucial to effective use of Org-roam.
+
+## File Aliases
+
+Suppose you want a note to be referred to by different names (e.g.
+"World War 2", "WWII"). You may specify such aliases using the
+`#+ROAM_ALIAS` attribute:
+
+```org
+#+TITLE: World War 2
+#+ROAM_ALIAS: "WWII" "World War II"
+```
+
+
+## File Refs
+
+Refs are unique identifiers for files. Each note can only have 1 ref.
+For example, a note for a website may contain a ref:
+
+```org
+#+TITLE: Google
+#+ROAM_KEY: https://www.google.com/
+```
+
+These keys come in useful for when taking website notes, using the
+`roam-ref` protocol (see [Roam Protocol](roam_protocol.md)).
+
+Alternatively, add a ref for notes for a specific paper, using its
+[org-ref](https://github.com/jkitchin/org-ref) citation key:
+
+```org
+#+TITLE: Neural Ordinary Differential Equations
+#+ROAM_KEY: cite:chen18_neural_ordin_differ_equat
+```
+
+In future, the backlinks buffer will also show notes with that
+citation key.

doc/comparison.md

@@ -1,7 +1,7 @@
 ---
-title: "Comparing Org-Roam With Other Packages"
-metaTitle: "Comparing Org-Roam With Other Packages"
-metaDescription: "Comparing Org-Roam With Other Packages"
+title: "Comparing Org-roam With Other Packages"
+metaTitle: "Comparing Org-roam With Other Packages"
+metaDescription: "Comparing Org-roam With Other Packages"
 ---
 
 # Org-brain

doc/configuration.md

@@ -0,0 +1,106 @@
+The number of configuration options is deliberately kept small, to
+keep the Org-roam codebase manageable. However, we attempt to
+accommodate as many usage styles as possible.
+
+All of Org-roam's customization options can be viewed via `M-x
+customize-group org-roam`.
+
+## Setting the Org-roam Directory
+
+Set `org-roam-directory` to the folder containing all your Org files:
+
+```emacs-lisp
+(setq org-roam-directory "/path/to/org/")
+```
+
+Every Org file, at any level of nesting, within `/path/to/org/` is
+considered part of the Org-roam ecosystem.
+
+### Having More Than One Org-roam Directory
+
+Emacs supports directory-local variables, allowing the value of
+`org-roam-directory` to be different in different directories. It does
+this by checking for a file named `.dir-locals.el`. 
+
+To add support for multiple directories, override the
+`org-roam-directory` variable using directory-local variables. This is
+what `.dir-locals.el` may contain:
+
+```emacs-lisp
+((nil . ((org-roam-directory . "/path/to/here/"))))
+```
+
+All files within that directory will be treated as their own separate
+set of Org-roam files. Remember to run `org-roam-build-cache` from a
+file within that directory, at least once.
+
+## Org-roam Buffer
+
+The Org-roam buffer defaults to popping up from the right. You may
+choose to set it to pop up from the left with `(setq
+org-roam-buffer-position 'left)`.
+
+The Org-roam buffer name can also be renamed: e.g. `(setq
+org-roam-buffer "*my-buffer-name*")`.
+
+The Org-roam buffer width is adjustable via `org-roam-buffer-width`.
+The value of `org-roam-buffer-width` set as a percentage of the total
+frame width. For example:
+
+```emacs-lisp
+(setq org-roam-buffer-width 0.4)
+```
+
+Will result in the Org-roam buffer taking up 40% of the screen width.
+
+## Org-roam Links
+
+By default, links are inserted with the title as the link description.
+This can make them hard to distinguish from external links. You may
+choose add special indicators for Org-roam links by tweaking
+`org-roam-link-title-format`, for example:
+
+```emacs-lisp
+(setq org-roam-link-title-format "R:%s")
+```
+
+If your version of Org is at least `9.2`, you may also choose to
+simply style the link differently, by customizing `org-roam-link-face`
+(`M-x customize-face org-roam-link`).
+
+## Org-roam Files
+
+Org-roam files are created and prefilled using Org-roam's templating
+system. The templating system is customizable, and the system is
+described in detail in the [Org-roam Template](templating.md) page.
+
+### Encryption
+
+Encryption (via GPG) can be enabled for all new files by setting
+`org-roam-encrypt-files` to `t`. When enabled, new files are created
+with the `.org.gpg` extension and decryption are handled automatically
+by EasyPG. 
+
+Note that Emacs will prompt for a password for encrypted files during
+cache updates if it requires reading the encrypted file. To reduce the
+number of password prompts, you may wish to cache the password.
+
+## Org-roam Graph Viewer
+
+Org-roam generates an SVG image using
+[Graphviz](https://graphviz.org/). To setup graph navigation, see the
+[Graph Setup](graph_setup.md) page.
+
+Org-roam tries its best to locate the Graphviz executable from your
+`PATH`, but if it fails to do so, you may set it manually:
+
+```
+(setq org-roam-graphviz-executable "/path/to/dot")
+```
+
+Org-roam also attempts to use Firefox (located on `PATH`) to view the
+SVG, you may choose to set it to any compatible program:
+
+```
+(setq org-roam-graph-viewer "/path/to/image-viewer")
+```

doc/ecosystem.md

@@ -1,12 +1,11 @@
-## Ecosystem
-
 A number of packages work well combined with Org-Roam:
 
-### Deft
-[Deft](https://jblevins.org/projects/deft/) provides a nice
-interface for browsing and filtering org-roam notes.
+## Deft
 
-```
+[Deft][deft] provides a nice interface for browsing and filtering
+org-roam notes.
+
+```emacs-lisp
 (use-package deft
   :after org
   :bind
@@ -15,17 +14,51 @@ interface for browsing and filtering org-roam notes.
   (deft-recursive t)
   (deft-use-filter-string-for-filename t)
   (deft-default-extension "org")
-  (deft-directory "/path/to/org-roam-files/")
-  (deft-use-filename-as-title t))
+  (deft-directory "/path/to/org-roam-files/"))
 ```
 
-### Org-journal
+If the title of the Org file is not the first line, you might not get
+nice titles. You may choose to patch this to use `org-roam`'s
+functionality. Here I'm using [el-patch](https://github.com/raxod502/el-patch):
+
+```emacs-lisp
+(use-package el-patch
+  :straight (:host github
+                   :repo "raxod502/el-patch"
+                   :branch "develop"))
+
+(eval-when-compile
+  (require 'el-patch))
+
+(use-package deft
+  ;; same as above...
+  :config/el-patch
+  (defun deft-parse-title (file contents)
+    "Parse the given FILE and CONTENTS and determine the title.
+If `deft-use-filename-as-title' is nil, the title is taken to
+be the first non-empty line of the FILE.  Else the base name of the FILE is
+used as title."
+    (el-patch-swap (if deft-use-filename-as-title
+                       (deft-base-filename file)
+                     (let ((begin (string-match "^.+$" contents)))
+                       (if begin
+                           (funcall deft-parse-title-function
+                                    (substring contents begin (match-end 0))))))
+                   (org-roam--get-title-or-slug file))))
+```
+
+The Deft interface can slow down quickly when the number of files get
+huge. [Notdeft][notdeft] is a fork of Deft that uses an external
+search engine and indexer.
+
+## Org-journal
+
 [Org-journal](https://github.com/bastibe/org-journal) is a more
 powerful alternative to the simple function `org-roam-today`. It
 provides better journaling capabilities, and a nice calendar interface
 to see all dated entries.
 
-```
+```emacs-lisp
 (use-package org-journal
   :bind
   ("C-c n j" . org-journal-new-entry)
@@ -35,3 +68,68 @@ to see all dated entries.
   (org-journal-dir "/path/to/org-roam-files/")
   (org-journal-date-format "%A, %d %B %Y"))
 ```
+
+## Note-taking Add-ons
+
+These are some plugins that make note-taking in Org-mode more
+enjoyable.
+
+### Org-download
+
+[Org-download][org-download] lets you screenshot and yank images from
+the web into your notes:
+
+![org-download](images/org-download.gif)
+
+```emacs-lisp
+(use-package org-download
+  :after org
+  :bind
+  (:map org-mode-map
+        (("s-Y" . org-download-screenshot)
+         ("s-y" . org-download-yank))))
+```
+
+### mathpix.el
+
+[mathpix.el][mathpix-el] uses [Mathpix's](https://mathpix.com/) API to convert clips into
+latex equations:
+
+![mathpix](images/mathpix.gif)
+
+```emacs-lisp
+(use-package mathpix.el
+  :straight (:host github :repo "jethrokuan/mathpix.el")
+  :custom ((mathpix-app-id "app-id")
+           (mathpix-app-key "app-key"))
+  :bind
+  ("C-x m" . mathpix-screenshot))
+```
+
+### Org-noter / Interleave
+
+[Org-noter][org-noter] and [Interleave][interleave] are both projects
+that allow synchronised annotation of documents (PDF, EPUB etc.)
+within Org-mode.
+
+### Org-ref
+
+[Org-ref][org-ref] does citation and bibliography management in
+Org-mode, and a great tool for scientific notes.
+
+### Spaced Repetition
+
+[Org-fc][org-fc] is a spaced repetition system that scales well with a
+large number of files. Other alternatives include
+[org-drill][org-drill], and [pamparam][pamparam].
+
+[deft]: https://jblevins.org/projects/deft/
+[notdeft]: https://github.com/hasu/notdeft
+[org-download]: https://github.com/abo-abo/org-download
+[mathpix-el]: https://github.com/jethrokuan/mathpix.el
+[org-noter]: https://github.com/weirdNox/org-noter
+[interleave]: https://github.com/rudolfochrist/interleave
+[org-ref]: https://github.com/jkitchin/org-ref
+[org-fc]: https://github.com/l3kn/org-fc/
+[org-drill]: https://orgmode.org/worg/org-contrib/org-drill.html
+[pamparam]: https://github.com/abo-abo/pamparam

doc/index.md

@@ -1,40 +1,62 @@
 ![org-roam][org-roam-intro-image]
 
-## What is Org-Roam?
+## What is Org-roam?
 
-Org-roam is a rudimentary [Roam][roamresearch] replica built around
-the all-powerful [Org-mode][org]. 
+Org-roam is a [Roam][roamresearch] replica built around the
+all-powerful [Org-mode][org].
 
-Like Roam, Org-roam offers a powerful and effortless non-hierarchical
-note-taking approach. With Org-roam, notes flow naturally, making
-note-taking fun and easy. To understand more about Roam, I recommend
-the following links:
+Org-roam is a solution for effortless non-hierarchical note-taking,
+building upon Org-mode. This solution is heavily inspired by [Roam
+Research][roamresearch]. With Org-roam, notes flow naturally, making
+note-taking fun and easy. Org-roam should also work as a plug-and-play
+solution for anyone already using Org-mode for their personal wiki.
 
--   [Building a second brain in
-    Roam](https://reddit.com/r/RoamResearch/comments/eho7de/building_a_second_brain_in_roamand_why_you_might)
--   [Roam: Why I Love It and How I Use
-    It](https://www.nateliason.com/blog/roam)
+To understand more about Roam, a collection of links are available in
+[the appendix](notetaking_workflow.md).
 
-The goal of the project is to implement core features of Roam around
-Org-mode, and eventually introduce features enabled by the Emacs
-ecosystem. 
+Org-roam aims to implement the core features of Roam, leveraging the
+mature ecosystem around Org-mode where possible. Eventually, we hope
+to further introduce features enabled by the Emacs ecosystem.
 
-## Why build Org-Roam?
+## Why use Org-roam?
 
-With Org-roam, you:
+##### Private and Secure
 
-1. Never have to leave Emacs.
-2. Can leverage all the powerful features of Org-mode: LaTeX, tables,
-   and the whole to-do ecosystem (org-agenda etc.)
-3. Be in full control your second brain, and access it offline. Never
-   share your data with anyone
+Edit your personal wiki completely offline, entirely in your control.
+Encrypt your notes with GPG.
+
+##### Longevity
+
+Unlike web solutions like Roam research, the notes are first and
+foremost plain Org-mode files -- Org-roam simply builds up an
+auxilliary database to give the personal wiki superpowers. Having your
+notes in plain-text is crucial for the longevity of your wiki. Never
+have to worry about proprietary web solutions being taken down. Edit
+your plain-text notes in notepad if all other editors cease to exist.
+
+##### Free and Open-source
+
+Org-roam is free and open-source, which means that if you feel unhappy
+with any part of Org-roam, you may choose to extend Org-roam, or open
+a PR.
+
+##### Leverage the Org-mode Ecosystem
+
+Over the years, Emacs and Org-mode has developed into a mature system
+for plain-text organization. Building upon Org-mode already puts
+Org-roam light-years ahead of many other solutions.
+
+Emacs is also a fantastic interface for editing text, and we can
+inherit many of the powerful text-navigation and editing packages
+available to Emacs.
 
 There are several packages that are similar to Org-roam, see the
 [Comparison](comparison.md) page for a detailed comparison.
 
 ## Project Status
 
-As of February 2020, it is in a very early stage of development. 
+As of March 2020, most of the core functionality and interfaces have
+stabilized, and a stable release is near.
 
 [org-roam-intro-image]: images/org-roam-intro.png
 [roamresearch]: https://www.roamresearch.com/

doc/installation.md

@@ -3,20 +3,20 @@
 The recommended method is using [use-package][use-package] and
 [straight][straight], or a similar package manager.
 
-```
+```emacs-lisp
 (use-package org-roam
-      :after org
-      :hook (org-mode . org-roam-mode)
-      :straight (:host github :repo "jethrokuan/org-roam")
+      :hook 
+      (after-init . org-roam-mode)
+      :straight (:host github :repo "jethrokuan/org-roam" :branch "develop")
       :custom
       (org-roam-directory "/path/to/org-files/")
-      (org-roam-link-representation 'title) ;; or keep it as 'id
-      :bind
-      ("C-c n l" . org-roam)      
-      ("C-c n t" . org-roam-today)
-      ("C-c n f" . org-roam-find-file)
-      ("C-c n i" . org-roam-insert)
-      ("C-c n g" . org-roam-show-graph))
+      :bind (:map org-roam-mode-map
+              (("C-c n l" . org-roam)
+               ("C-c n f" . org-roam-find-file)
+               ("C-c n b" . org-roam-switch-to-buffer)
+               ("C-c n g" . org-roam-show-graph))
+              :map org-mode-map
+              (("C-c n i" . org-roam-insert))))
 ```
 
 If not using package.el, you can also clone it into your Emacs
@@ -26,10 +26,108 @@ directory and add it to your load path:
 git clone https://github.com/jethrokuan/org-roam/ ~/.emacs.d/elisp/org-roam
 ```
 
+```emacs-lisp
+(use-package org-roam
+  :load-path "elisp/"
+  :hook
+  (after-init . org-roam-mode)
+  :straight (:host github :repo "jethrokuan/org-roam" :branch "develop")
+  :custom
+  (org-roam-directory "/path/to/org-files/")
+  :bind (:map org-roam-mode-map
+              (("C-c n l" . org-roam)
+               ("C-c n f" . org-roam-find-file)
+               ("C-c n b" . org-roam-switch-to-buffer)
+               ("C-c n g" . org-roam-show-graph))
+              :map org-mode-map
+              (("C-c n i" . org-roam-insert))))
 ```
+
+Or without `use-package`:
+
+```emacs-lisp
 (add-to-list 'load-path "./elisp")
 (require 'org-roam)
+(define-key 'org-roam-mode-map (kbd "C-c n l") #'org-roam)
+(define-key 'org-roam-mode-map (kbd "C-c n f") #'org-roam-find-file)
+(define-key 'org-roam-mode-map (kbd "C-c n b") #'org-roam-switch-to-buffer)
+(define-key 'org-roam-mode-map (kbd "C-c n g") #'org-roam-switch-to-buffer)
+(define-key 'org-mode-map (kbd "C-c n i") #'org-roam-insert)
+(org-roam-mode +1)
 ```
 
+The [Configuration](configuration.md) page details some of the common
+configuration options available.
+
 [use-package]: https://github.com/jwiegley/use-package
 [straight]: https://github.com/raxod502/straight.el
+
+## Spacemacs
+If you are using Spacemacs, install org-roam by creating a simple
+layer that wraps Org-roam. Paste the following into a new file
+`~/.emacs.d/private/org-roam/packages.el`.
+
+```emacs-lisp
+(defconst org-roam-packages
+  '((org-roam :location
+        (recipe :fetcher github :repo "jethrokuan/org-roam" :branch "develop"))))
+
+(defun org-roam/init-org-roam ()
+    (use-package org-roam
+        :hook
+        (after-init . org-roam-mode)
+        :custom
+        (org-roam-directory "/path/to/org-files/")
+        :init
+        (progn
+          (spacemacs/declare-prefix "ar" "org-roam")
+          (spacemacs/set-leader-keys
+            "arl" 'org-roam
+            "art" 'org-roam-today
+            "arf" 'org-roam-find-file
+            "arg" 'org-roam-show-graph)
+
+          (spacemacs/declare-prefix-for-mode 'org-mode "mr" "org-roam")
+          (spacemacs/set-leader-keys-for-major-mode 'org-mode
+            "rl" 'org-roam
+            "rt" 'org-roam-today
+            "rb" 'org-roam-switch-to-buffer
+            "rf" 'org-roam-find-file
+            "ri" 'org-roam-insert
+            "rg" 'org-roam-show-graph))))
+```
+
+Next, append `org-roam` to the `dotspacemacs-configuration-layers`
+list in your `.spacemacs` configuration file. Reload (`SPC f e R`) or
+restart Emacs to load `org-roam`. It's functions are available under
+the prefix `SPC a r` and `, r` when visiting an org-mode buffer.
+
+## Doom Emacs
+
+If you are using [Doom Emacs](https://github.com/hlissner/doom-emacs), configure packages as explained in the [getting started](https://github.com/hlissner/doom-emacs/blob/develop/docs/getting_started.org#configuring-packages) guide. 
+
+Declare Org-roam as a package in your `~/.doom.d/packages.el`:
+
+```elisp
+;; ~/.doom.d/packages.el
+
+(package! org-roam
+  :recipe (:host github :repo "jethrokuan/org-roam" :branch "develop"))
+```
+
+Subsequently, in your `~/.doom.d/config.el` file, configure Org-roam:
+
+```elisp
+;; ~/.doom.d/config.el
+(use-package! org-roam
+  :commands (org-roam-insert org-roam-find-file org-roam)
+  :init 
+  (setq org-roam-directory "/path/to/org-files/")
+  (map! :leader 
+        :prefix "n"
+        :desc "Org-Roam-Insert" "i" #'org-roam-insert
+        :desc "Org-Roam-Find"   "/" #'org-roam-find-file
+        :desc "Org-Roam-Buffer" "r" #'org-roam)
+  :config
+  (org-roam-mode +1))
+```

doc/notetaking_workflow.md

@@ -0,0 +1,24 @@
+## Recommended Books
+- [How to Take Smart Notes][1]
+
+## Articles
+- [How to Take Smart Notes in Org-mode - Jethro Kuan][7]
+- [The Zettelkasten Method - LessWrong 2.0][3]
+- [Building a second brain in Roam][4]
+- [Roam: Why I Love It and How I Use It][5]
+- [Adam Keesling's Twitter Thread][6]
+
+## Threads
+- [Ask HN: How to Take Good Notes][8]
+
+## What to Do With Your Notes
+- [How to Use Roam to Outline a New Article in Under 20 Minutes][2]
+
+[1]: https://www.goodreads.com/book/show/34507927-how-to-take-smart-notes?ac=1&from_search=true&qid=6L8iEE1FIA&rank=1
+[2]: https://www.youtube.com/watch?v=RvWic15iXjk
+[3]: https://www.lesswrong.com/posts/NfdHG6oHBJ8Qxc26s/the-zettelkasten-method-1
+[4]: https://reddit.com/r/RoamResearch/comments/eho7de/building_a_second_brain_in_roamand_why_you_might
+[5]: https://www.nateliason.com/blog/roam
+[6]: https://twitter.com/adam_keesling/status/1196864424725774336?s=20
+[7]: https://blog.jethro.dev/posts/how_to_take_smart_notes_org/
+[8]: https://news.ycombinator.com/item?id=22473209

doc/roam_protocol.md

@@ -0,0 +1,161 @@
+## What is Roam protocol?
+
+Org-roam extending `org-protocol` with 2 protocols: the `roam-file`
+and `roam-ref` protocol.
+
+## The `roam-file` protocol
+
+This is a simple protocol that opens the path specified by the `file`
+key (e.g. `org-protocol://roam-file?file=/tmp/file.org`). This is used
+in the generated graph.
+
+## The `roam-ref` Protocol
+
+This protocol finds or creates a new note with a given `ROAM_KEY` (see
+[Anatomy](anatomy.md)):
+
+![roam-ref](images/roam-ref.gif)
+
+To use this, create a Firefox bookmarklet as follows:
+
+```javascript
+javascript:location.href =
+'org-protocol:/roam-ref?template=r&ref='
++ encodeURIComponent(location.href)
++ '&title='
++ encodeURIComponent(document.title)
+```
+
+where `template` is the template key for a template in
+`org-roam-ref-capture-templates`. More documentation on the templating
+system can be found [here](templating.md).
+
+These templates should contain a `#+ROAM_KEY: ${ref}` in it.
+
+## Setting up Org-roam protocol
+
+To enable org-roam's protocol extensions, you have to add the
+following to your init file:
+
+```emacs-lisp
+(require 'org-roam-protocol)
+```
+
+The instructions for setting up `org-protocol` can be found
+[here][org-protocol-inst], but they are reproduced below.
+
+We will also need to create a desktop application for `emacsclient`.
+The instructions for various platforms are shown below:
+
+## Linux
+
+Create a desktop application. I place mine in
+`~/.local/share/applications/org-protocol.desktop`:
+
+```
+[Desktop Entry]
+Name=Org-Protocol
+Exec=emacsclient %u
+Icon=emacs-icon
+Type=Application
+Terminal=false
+MimeType=x-scheme-handler/org-protocol
+```
+
+Associate `org-protocol://` links with the desktop application by
+running in your shell:
+
+```bash
+xdg-mime default org-protocol.desktop x-scheme-handler/org-protocol
+```
+
+To disable the "confirm" prompt in Chrome, you can also make Chrome
+show a checkbox to tick, so that the `Org-Protocol Client` app will be used
+without confirmation. To do this, run in a shell:
+
+```sh
+sudo mkdir -p /etc/opt/chrome/policies/managed/
+sudo tee /etc/opt/chrome/policies/managed/external_protocol_dialog.json >/dev/null <<'EOF'
+{
+  "ExternalProtocolDialogShowAlwaysOpenCheckbox": true
+}
+EOF
+sudo chmod 644 /etc/opt/chrome/policies/managed/external_protocol_dialog.json
+```
+
+and then restart Chrome (for example, by navigating to <chrome://restart>) to
+make the new policy take effect.
+
+See [here](https://www.chromium.org/administrators/linux-quick-start)
+for more info on the `/etc/opt/chrome/policies/managed` directory and
+[here](https://cloud.google.com/docs/chrome-enterprise/policies/?policy=ExternalProtocolDialogShowAlwaysOpenCheckbox)
+for information on the `ExternalProtocolDialogShowAlwaysOpenCheckbox`
+policy.
+
+
+## Mac OS
+
+One solution is to use
+[Platypus](https://github.com/sveinbjornt/Platypus). Here are the
+instructions for setting up with Platypus and Chrome:
+
+1. Install and launch Platypus (with [Homebrew](https://brew.sh/)):
+
+```sh
+brew cask install playtpus
+```
+2. Create a script `launch_emacs.sh`:
+
+```
+#!/usr/bin/env bash
+/usr/local/bin/emacsclient --no-wait $1
+```
+
+3. Create a Platypus app with the following settings:
+
+```
+| Setting                        | Value                     |
+|--------------------------------+---------------------------|
+| App Name                       | "OrgProtocol"             |
+| Script Type                    | "env" · "/usr/bin/env"    |
+| Script Path                    | "path/to/launch-emacs.sh" |
+| Interface                      | None                      |
+| Accept dropped items           | true                      |
+| Remain running after execution | false                     |
+```
+
+Inside `Settings`:
+
+```
+| Setting                        | Value          |
+|--------------------------------+----------------|
+| Accept dropped files           | true           |
+| Register as URI scheme handler | true           |
+| Protocol                       | "org-protocol" |
+```
+
+To disable the "confirm" prompt in Chrome, you can also make Chrome
+show a checkbox to tick, so that the `OrgProtocol` app will be used
+without confirmation. To do this, run in a shell:
+
+```sh
+defaults write com.google.Chrome ExternalProtocolDialogShowAlwaysOpenCheckbox -bool true
+```
+
+
+##### Note for Emacs Mac Port
+
+If you're using [Emacs Mac Port](https://github.com/railwaycat/homebrew-emacsmacport), it
+registered its `Emacs.app` as the default handler for the URL scheme
+`org-protocol`. We have to make our `OrgProtocol.app` the default
+handler instead (replace `org.yourusername.OrgProtocol` with your app
+identifier):
+
+```
+$ defaults write com.apple.LaunchServices/com.apple.launchservices.secure LSHandlers -array-add \
+'{"LSHandlerPreferredVersions" = { "LSHandlerRoleAll" = "-"; }; LSHandlerRoleAll = "org.yourusername.OrgProtocol"; LSHandlerURLScheme = "org-protocol";}'
+```
+
+Then restart your computer.
+
+[org-protocol-inst]: https://orgmode.org/worg/org-contrib/org-protocol.html

doc/templating.md

@@ -0,0 +1,83 @@
+Rather than creating blank files on `org-roam-insert` and
+`org-roam-find-file`, it is may be desirable to prefill the file with
+content. This may include:
+
+- Time of creation
+- File it was created from
+- Clipboard content
+- Any other data you may want to input manually
+
+This requires a complex template insertion system, but fortunately,
+Org ships with a powerful one: `org-capture`. However, org-capture was
+not designed for such use. Org-roam abuses `org-capture` to some
+extent, extending its syntax. To first understand how org-roam's
+templating system works, it may be useful to look into org-capture.
+
+## Org-roam Templates
+
+The org-roam capture template extends org-capture's template with 2
+additional properties:
+
+1. `:file-name`: This is the file name template used when a new note
+   is created.
+2. `:head`: This is the template that is inserted on initial note
+   creation. 
+
+### Org-roam Template Expansion
+
+Org-roam's template definitions also extend org-capture's template
+syntax, to allow prefilling of strings. In many scenarios,
+`org-roam-capture` is passed a mapping between variables and strings.
+For example, during `org-roam-insert`, a title is prompted for. If the
+title doesn't already exist, we would like to create a new file,
+without prompting for the title again.
+
+Variables passed are expanded with the `${var}` syntax. For example,
+eduring `org-roam-insert`, `${title}` is prefilled for expansion. Any
+variables that do not contain strings, are prompted for values using
+`completing-read`.
+
+After doing this expansion, the org-capture's template expansion
+system is used to fill up the rest of the template. You may read up
+more on this on [org-capture's documentation
+page](https://orgmode.org/manual/Template-expansion.html#Template-expansion).
+
+For example, take the template: `"%<%Y%m%d%H%M%S>-${title}"`, with the title
+`"Foo"`. The template is first expanded into `%<%Y%m%d%H%M%S>-Foo`. Then
+org-capture expands `%<%Y%m%d%H%M%S>` with timestamp: e.g.
+`20200213032037-Foo`.
+
+This templating system is used throughout org-roam templates.
+
+### Template examples
+
+Here I walkthrough the default template, reproduced below.
+
+```
+("d" "default" plain (function org-roam--capture-get-point)
+     "%?"
+     :file-name "%<%Y%m%d%H%M%S>-${slug}"
+     :head "#+TITLE: ${title}\n"
+     :unnarrowed t)
+```
+
+1. The template has short key `"d"`. If you have only one template,
+   org-roam automatically chooses this template for you.
+2. The template is given a description of `"default"`.
+3. `plain` text is inserted. Other options include Org headings via
+   `entry`.
+4. `(function org-roam--capture-get-point)` should not be changed.
+5. `"%?"` is the template inserted on each call to `org-roam-capture`.
+   This template means don't insert any content, but place the cursor
+   here.
+6. `:file-name` is the file-name template for a new note, if it
+doesn't yet exist. This creates a file at path that looks like
+   `/path/to/org-roam-directory/20200213032037-foo.org`.
+7. `:head` contains the initial template to be inserted (once only),
+   at the beginning of the file. Here, the title global attribute is
+   inserted.
+8. `:unnarrowed t` tells org-capture to show the contents for the
+   whole file, rather than narrowing to just the entry.
+
+Other options you may want to learn about include `:immediate-finish`.
+

doc/tour.md

@@ -1,20 +1,65 @@
-### A Tour of Org-Roam
+Org-roam was built to support a workflow that was not possible with
+vanilla Org-mode. This flow is modelled after the [Zettelkasten
+method][zettelkasten], and many of [Roam Research][roam]'s workflows.
+It is crucial to understand that Org-roam does not auto-magically make
+note-taking better -- it's changing the note-taking workflow that
+does.
 
-All of this starts from the note. A note is just a simple `.org` file
-in the directory. Any org file in the directory is considered part of
-the org-roam ecosystem. Notes are quickly linked together (and created
-if necessary) using `org-roam-insert`.
+To understand more about the methods and madness, the [Note-Taking
+Workflow][appendix:ntw] page contains a page of useful references. The
+author has also written [a post][jethro-blog-post] about how he uses
+Org-roam.
 
-![org-roam-insert](images/org-roam-insert.gif)
+## Activating Org-roam
 
-Org-roam tracks all of these file links, and builds a cache
-asynchronously in the background. This cache is used to populate the
-backlinks buffer, which shows files that link to the current file, as
-well as some preview contents:
+Org-roam's entry point is the global minor `org-roam-mode`. This sets
+up Emacs with several hooks, for keeping the org-roam cache
+consistently updated, as well as showing the backlinks buffer. 
+
+The cache is a sqlite database named `org-roam.db`, which resides at
+the root of the `org-roam-directory`. Activating `org-roam-mode`
+builds the cache, which may take a while the first time, but is
+generally instantaneous in subsequent runs. To build the cache
+manually again, run `M-x org-roam-build-cache`.
+
+## Finding a Note
+
+`org-roam-find-file` shows the list of titles for notes that reside in
+`org-roam-directory`. Selecting a note title will bring you to the
+corresponding note. Entering a title of a note that does not yet exist
+will create a new note with that title.
+
+![org-roam-find-file](images/org-roam-find-file.gif)
+
+## Inserting Links
+
+`org-roam-insert` insert links to existing (or new) notes. Entering a
+non-existent title will also create a new note with that title.
+
+![org-roam-insert](images/org-roam-insert-filetag.gif)
+
+Good usage of Org-roam requires liberally linking files. This allows
+the build-up of a dense knowledge graph.
+
+## The Org-roam Buffer
+
+The Org-roam buffer is often displayed in the side window. It shows
+backlinks for the currently active Org-roam note, along with some
+surrounding context.
 
 ![org-roam-buffer](images/org-roam-buffer.gif)
 
-These file links also form a graph. The generated graph is navigable
-in Emacs.
+## Exporting the Graph
+
+Org-roam also uses Graphviz to generate a graph, with notes as nodes,
+and links between them as edges. The generated graph can be used to
+navigate to the files, but this requires some additional setup
+described in the [Roam Protocol][appendix:roam-protocol] page.
 
 ![org-roam-graph](images/org-roam-graph.gif)
+
+[zettelkasten]: https://zettelkasten.de/
+[appendix:ntw]: notetaking_workflow.md
+[appendix:roam-protocol]: roam_protocol.md
+[roam]: https://www.roamresearch.com/
+[jethro-blog-post]: https://blog.jethro.dev/posts/how_to_take_smart_notes_org/

mkdocs.yml

@@ -1,14 +1,19 @@
-site_name: "Org-Roam: Roam + Org-Mode = ♥"
+site_name: "Org-roam"
 repo_url: https://github.com/jethrokuan/org-roam/
 edit_uri: edit/master/doc/
 copyright: "Copyright (C) 2020 Jethro Kuan and contributors"
 docs_dir: doc
 nav:
 - Home: index.md
-- A Tour of Org-Roam: tour.md
+- A Tour of Org-roam: tour.md
 - Installation: installation.md
+- Configuration: configuration.md
+- Anatomy of an Org-roam file: anatomy.md
+- The Templating System: templating.md
 - Ecosystem: ecosystem.md
 - Similar Packages: comparison.md
+- "Appendix: Note-taking Workflow": notetaking_workflow.md
+- "Appendix: Roam Protocol": roam_protocol.md
 markdown_extensions:
   - admonition
   - pymdownx.betterem:

org-roam-protocol.el

@@ -0,0 +1,103 @@
+;;; org-roam-protocol.el --- Protocol handler for roam:// links  -*- coding: utf-8; lexical-binding: t -*-
+
+;; Copyright © 2020 Jethro Kuan <jethrokuan95@gmail.com>
+;; Author: Jethro Kuan <jethrokuan95@gmail.com>
+;; URL: https://github.com/jethrokuan/org-roam
+;; Keywords: org-mode, roam, convenience
+;; Version: 1.0.0-rc1
+;; Package-Requires: ((emacs "26.1") (org "9.0"))
+
+;; This file is NOT part of GNU Emacs.
+
+;; This program is free software; you can redistribute it and/or modify
+;; it under the terms of the GNU General Public License as published by
+;; the Free Software Foundation; either version 3, or (at your option)
+;; any later version.
+;;
+;; This program is distributed in the hope that it will be useful,
+;; but WITHOUT ANY WARRANTY; without even the implied warranty of
+;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+;; GNU General Public License for more details.
+;;
+;; You should have received a copy of the GNU General Public License
+;; along with GNU Emacs; see the file COPYING.  If not, write to the
+;; Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor,
+;; Boston, MA 02110-1301, USA.
+
+;;; Commentary:
+;;
+;; We extend org-protocol, adding custom Org-roam handlers. The setup
+;; instructions for `org-protocol' can be found in org-protocol.el.
+;;
+;; We define 2 protocols:
+;;
+;; 1. "roam-file": This protocol simply opens the file given by the FILE key
+;; 2. "roam-ref": This protocol creates or opens a note with the given REF
+;;
+;;; Code:
+
+(require 'org-protocol)
+(require 'org-roam)
+
+(declare-function org-roam-find-ref "org-roam" (&optional info))
+(declare-function org-roam--capture-get-point "org-roam" ())
+
+(defvar org-roam-ref-capture-templates
+  '(("r" "ref" plain (function org-roam--capture-get-point)
+     ""
+     :file-name "${slug}"
+     :head "#+TITLE: ${title}
+#+ROAM_KEY: ${ref}\n"
+     :unnarrowed t))
+  "The Org-roam templates used during a capture from the roam-ref protocol.
+Details on how to specify for the template is given in `org-roam-capture-templates'.")
+
+(defun org-roam-protocol-open-ref (info)
+  "Process an org-protocol://roam-ref?ref= style url with INFO.
+
+It opens or creates a note with the given ref.
+
+  javascript:location.href = \\='org-protocol://roam-ref?template=r&ref=\\='+ \\
+        encodeURIComponent(location.href) + \\='&title=\\=' \\
+        encodeURIComponent(document.title) + \\='&body=\\=' + \\
+        encodeURIComponent(window.getSelection())"
+  (when-let* ((alist (org-roam--plist-to-alist info))
+              (decoded-alist (mapcar (lambda (k.v)
+                                       (let ((key (car k.v))
+                                             (val (cdr k.v)))
+                                         (cons key (org-link-decode val)))) alist)))
+    (unless (assoc 'ref decoded-alist)
+      (error "No ref key provided."))
+    (when-let ((title (cdr (assoc 'title decoded-alist))))
+      (push (cons 'slug (org-roam--title-to-slug title)) decoded-alist))
+    (let* ((org-roam-capture-templates org-roam-ref-capture-templates)
+           (org-roam--capture-context 'ref)
+           (org-roam--capture-info decoded-alist)
+           (template (cdr (assoc 'template decoded-alist))))
+      (raise-frame)
+      (org-roam-capture nil template)
+      (message "Item captured.")))
+  nil)
+
+(defun org-roam-protocol-open-file (info)
+  "This handler simply opens the file with emacsclient.
+
+INFO is an alist containing additional information passed by the protocol URL.
+It should contain the FILE key, pointing to the path of the file to open.
+
+  Example protocol string:
+
+org-protocol://roam-file?file=/path/to/file.org"
+  (when-let ((file (plist-get info :file)))
+    (raise-frame)
+    (find-file file))
+  nil)
+
+(push '("org-roam-ref"  :protocol "roam-ref"   :function org-roam-protocol-open-ref)
+      org-protocol-protocol-alist)
+(push '("org-roam-file"  :protocol "roam-file"   :function org-roam-protocol-open-file)
+      org-protocol-protocol-alist)
+
+(provide 'org-roam-protocol)
+
+;;; org-roam-protocol.el ends here

shell.nix

@@ -1,10 +0,0 @@
-let
-  pkgs = import <nixpkgs> {};
-in
-pkgs.mkShell {
-  name = "docs";
-  buildInput = with pkgs; [
-    mkdocs
-    python3Packages.alabaster
-  ];
-}

tests/roam-files/alias.org

@@ -0,0 +1,2 @@
+#+ROAM_ALIAS: "a1" "a 2"
+#+TITLE: t1

tests/roam-files/bar.org

@@ -0,0 +1,3 @@
+#+TITLE: Bar
+
+This is file bar. Bar links to [[file:nested/bar.org][Nested Bar]].

tests/roam-files/foo.org

@@ -0,0 +1,8 @@
+#+TITLE: Foo
+
+This is the foo file. It contains a link to [[file:bar.org][Bar]].
+
+To make the tests more robust, here are some arbitrary links:
+
+- [[https:google.com][Google]]
+- [[mailto:foo@john.com][mail to foo]]

tests/roam-files/nested/bar.org

@@ -0,0 +1,3 @@
+#+TITLE: Nested Bar
+
+This file is nested, 1 level deeper. It links to both [[file:../foo.org][Foo]] and [[file:foo.org][Nested Foo]].

tests/roam-files/nested/foo.org

@@ -0,0 +1,3 @@
+#+TITLE: Nested Foo
+
+This file has no links.

tests/roam-files/no-title.org

@@ -0,0 +1,3 @@
+no title in this file :O
+
+links to itself, with no title: [[file:no-title.org][no-title]]

tests/roam-files/unlinked.org

@@ -0,0 +1,3 @@
+#+TITLE: Unlinked
+
+Nothing links here :(

tests/roam-files/web_ref.org

@@ -0,0 +1 @@
+#+ROAM_KEY: https://google.com/

tests/test-org-roam.el

@@ -0,0 +1,303 @@
+;;; test-org-roam.el --- Tests for org-roam -*- lexical-binding: t; -*-
+
+;; Copyright (C) 2020 Jethro Kuan
+
+;; Author: Jethro Kuan <jethrokuan95@gmail.com>
+;; Package-Requires: ((buttercup) (with-simulated-input))
+
+;; This program is free software; you can redistribute it and/or modify
+;; it under the terms of the GNU General Public License as published by
+;; the Free Software Foundation, either version 3 of the License, or
+;; (at your option) any later version.
+
+;; This program is distributed in the hope that it will be useful,
+;; but WITHOUT ANY WARRANTY; without even the implied warranty of
+;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+;; GNU General Public License for more details.
+
+;; You should have received a copy of the GNU General Public License
+;; along with this program.  If not, see <https://www.gnu.org/licenses/>.
+
+;;; Commentary:
+
+;;
+
+;;; Code:
+
+;;;; Requirements
+
+(require 'buttercup)
+(require 'with-simulated-input)
+(require 'org-roam)
+(require 'dash)
+
+(defun abs-path (file-path)
+  (file-truename (expand-file-name file-path org-roam-directory)))
+
+(defun org-roam--test-find-new-file (path)
+  (let ((path (abs-path path)))
+    (make-directory (file-name-directory path) t)
+    (find-file path)))
+
+(defvar org-roam--tests-directory (file-truename (concat default-directory "tests/roam-files"))
+  "Directory containing org-roam test org files.")
+
+(defun org-roam--test-init ()
+  (org-roam--db-close)
+  (let ((original-dir org-roam--tests-directory)
+        (new-dir (expand-file-name (make-temp-name "org-roam") temporary-file-directory)))
+    (copy-directory original-dir new-dir)
+    (setq org-roam-directory new-dir)
+    (org-roam-mode +1)))
+
+;;; Tests
+(describe "org-roam-build-cache"
+          (it "initializes correctly"
+              (org-roam--test-init)
+              (org-roam-build-cache)
+
+              ;; Cache
+              (expect (caar (org-roam-sql [:select (funcall count) :from files])) :to-be 8)
+              (expect (caar (org-roam-sql [:select (funcall count) :from file-links])) :to-be 5)
+              (expect (caar (org-roam-sql [:select (funcall count) :from titles])) :to-be 8)
+              (expect (caar (org-roam-sql [:select (funcall count) :from titles
+                                                   :where titles :is-null])) :to-be 2)
+              (expect (caar (org-roam-sql [:select (funcall count) :from refs])) :to-be 1)
+
+              ;; TODO Test files
+
+              ;; Links -- File-from
+              (expect (caar (org-roam-sql [:select (funcall count) :from file-links
+                                                   :where (= file-from $s1)]
+                                          (abs-path "foo.org"))) :to-be 1)
+              (expect (caar (org-roam-sql [:select (funcall count) :from file-links
+                                                   :where (= file-from $s1)]
+                                          (abs-path "nested/bar.org"))) :to-be 2)
+
+              ;; Links -- File-to
+              (expect (caar (org-roam-sql [:select (funcall count) :from file-links
+                                                   :where (= file-to $s1)]
+                                          (abs-path "nested/foo.org"))) :to-be 1)
+              (expect (caar (org-roam-sql [:select (funcall count) :from file-links
+                                                   :where (= file-to $s1)]
+                                          (abs-path "nested/bar.org"))) :to-be 1)
+              (expect (caar (org-roam-sql [:select (funcall count) :from file-links
+                                                   :where (= file-to $s1)]
+                                          (abs-path "unlinked.org"))) :to-be 0)
+              ;; TODO Test titles
+              (expect (org-roam-sql [:select * :from titles])
+                      :to-have-same-items-as
+                      (list (list (abs-path "alias.org")
+                                  (list "t1" "a1" "a 2"))
+                            (list (abs-path "bar.org")
+                                  (list "Bar"))
+                            (list (abs-path "foo.org")
+                                  (list "Foo"))
+                            (list (abs-path "nested/bar.org")
+                                  (list "Nested Bar"))
+                            (list (abs-path "nested/foo.org")
+                                  (list "Nested Foo"))
+                            (list (abs-path "no-title.org") nil)
+                            (list (abs-path "web_ref.org") nil)
+                            (list (abs-path "unlinked.org")
+                                  (list "Unlinked"))))
+
+              (expect (org-roam-sql [:select * :from refs])
+                      :to-have-same-items-as
+                      (list (list "https://google.com/" (abs-path "web_ref.org"))))
+
+              ;; Expect rebuilds to be really quick (nothing changed)
+              (expect (org-roam-build-cache)
+                      :to-equal
+                      (list :files 0 :links 0 :titles 0 :refs 0 :deleted 0))))
+
+(describe "org-roam-insert"
+          (before-each
+           (org-roam--test-init)
+           (org-roam--db-clear)
+           (org-roam-build-cache))
+
+          (it "temp1 -> foo"
+              (let ((buf (org-roam--test-find-new-file "temp1.org")))
+                (with-current-buffer buf
+                  (with-simulated-input
+                   "Foo RET"
+                   (org-roam-insert nil))))
+              (expect (buffer-string) :to-match (regexp-quote "file:foo.org")))
+
+          (it "temp2 -> nested/foo"
+              (let ((buf (org-roam--test-find-new-file "temp2.org")))
+                (with-current-buffer buf
+                  (with-simulated-input
+                   "Nested SPC Foo RET"
+                   (org-roam-insert nil))))
+              (expect (buffer-string) :to-match (regexp-quote "file:nested/foo.org")))
+
+          (it "nested/temp3 -> foo"
+              (let ((buf (org-roam--test-find-new-file "nested/temp3.org")))
+                (with-current-buffer buf
+                  (with-simulated-input
+                   "Foo RET"
+                   (org-roam-insert nil))))
+              (expect (buffer-string) :to-match (regexp-quote "file:../foo.org")))
+
+          (it "a/b/temp4 -> nested/foo"
+              (let ((buf (org-roam--test-find-new-file "a/b/temp4.org")))
+                (with-current-buffer buf
+                  (with-simulated-input
+                   "Nested SPC Foo RET"
+                   (org-roam-insert nil))))
+              (expect (buffer-string) :to-match (regexp-quote "file:../../nested/foo.org"))))
+
+(describe "rename file updates cache"
+          (before-each
+           (org-roam--test-init)
+           (org-roam--db-clear)
+           (org-roam-build-cache))
+
+          (it "foo -> new_foo"
+              (rename-file (abs-path "foo.org")
+                           (abs-path "new_foo.org"))
+              ;; Cache should be cleared of old file
+              (expect (caar (org-roam-sql [:select (funcall count)
+                                           :from titles
+                                           :where (= file $s1)]
+                                          (abs-path "foo.org"))) :to-be 0)
+              (expect (caar (org-roam-sql [:select (funcall count)
+                                           :from refs
+                                           :where (= file $s1)]
+                                          (abs-path "foo.org"))) :to-be 0)
+              (expect (caar (org-roam-sql [:select (funcall count)
+                                           :from file-links
+                                           :where (= file-from $s1)]
+                                          (abs-path "foo.org"))) :to-be 0)
+
+              ;; Cache should be updated
+              (expect (org-roam-sql [:select [file-to]
+                                     :from file-links
+                                     :where (= file-from $s1)]
+                                    (abs-path "new_foo.org"))
+                      :to-have-same-items-as
+                      (list (list (abs-path "bar.org"))))
+              (expect (org-roam-sql [:select [file-from]
+                                     :from file-links
+                                     :where (= file-to $s1)]
+                                    (abs-path "new_foo.org"))
+                      :to-have-same-items-as
+                      (list (list (abs-path "nested/bar.org"))))
+
+              ;; Links are updated
+              (expect (with-temp-buffer
+                        (insert-file-contents (abs-path "nested/bar.org"))
+                        (buffer-string))
+                      :to-match
+                      (regexp-quote "[[file:../new_foo.org][Foo]]")))
+
+          (it "foo -> foo with spaces"
+              (rename-file (abs-path "foo.org")
+                           (abs-path "foo with spaces.org"))
+              ;; Cache should be cleared of old file
+              (expect (caar (org-roam-sql [:select (funcall count)
+                                           :from titles
+                                           :where (= file $s1)]
+                                          (abs-path "foo.org"))) :to-be 0)
+              (expect (caar (org-roam-sql [:select (funcall count)
+                                           :from refs
+                                           :where (= file $s1)]
+                                          (abs-path "foo.org"))) :to-be 0)
+              (expect (caar (org-roam-sql [:select (funcall count)
+                                           :from file-links
+                                           :where (= file-from $s1)]
+                                          (abs-path "foo.org"))) :to-be 0)
+
+              ;; Cache should be updated
+              (expect (org-roam-sql [:select [file-to]
+                                     :from file-links
+                                     :where (= file-from $s1)]
+                                    (abs-path "foo with spaces.org"))
+                      :to-have-same-items-as
+                      (list (list (abs-path "bar.org"))))
+              (expect (org-roam-sql [:select [file-from]
+                                     :from file-links
+                                     :where (= file-to $s1)]
+                                    (abs-path "foo with spaces.org"))
+                      :to-have-same-items-as
+                      (list (list (abs-path "nested/bar.org"))))
+
+              ;; Links are updated
+              (expect (with-temp-buffer
+                        (insert-file-contents (abs-path "nested/bar.org"))
+                        (buffer-string))
+                      :to-match
+                      (regexp-quote "[[file:../foo with spaces.org][Foo]]")))
+
+          (it "no-title -> meaningful-title"
+              (rename-file (abs-path "no-title.org")
+                           (abs-path "meaningful-title.org"))
+              ;; File has no forward links
+              (expect (caar (org-roam-sql [:select (funcall count)
+                                           :from file-links
+                                           :where (= file-from $s1)]
+                                          (abs-path "no-title.org"))) :to-be 0)
+              (expect (caar (org-roam-sql [:select (funcall count)
+                                           :from file-links
+                                           :where (= file-from $s1)]
+                                          (abs-path "meaningful-title.org"))) :to-be 1)
+
+              ;; Links are updated with the appropriate name
+              (expect (with-temp-buffer
+                        (insert-file-contents (abs-path "meaningful-title.org"))
+                        (buffer-string))
+                      :to-match
+                      (regexp-quote "[[file:meaningful-title.org][meaningful-title]]")))
+
+          (it "web_ref -> hello"
+              (expect (org-roam-sql
+                       [:select [file] :from refs
+                        :where (= ref $s1)]
+                       "https://google.com/")
+                      :to-equal
+                      (list (list (abs-path "web_ref.org"))))
+              (rename-file (abs-path "web_ref.org")
+                           (abs-path "hello.org"))
+              (expect (org-roam-sql
+                       [:select [file] :from refs
+                        :where (= ref $s1)]
+                       "https://google.com/")
+                      :to-equal (list (list (abs-path "hello.org"))))
+              (expect (caar (org-roam-sql
+                             [:select [ref] :from refs
+                              :where (= file $s1)]
+                             (abs-path "web_ref.org")))
+                      :to-equal nil)))
+
+(describe "delete file updates cache"
+          (before-each
+           (org-roam--test-init)
+           (org-roam--db-clear)
+           (org-roam-build-cache)
+           (sleep-for 1))
+
+          (it "delete foo"
+              (delete-file (abs-path "foo.org"))
+              (expect (caar (org-roam-sql [:select (funcall count)
+                                           :from titles
+                                           :where (= file $s1)]
+                                          (abs-path "foo.org"))) :to-be 0)
+              (expect (caar (org-roam-sql [:select (funcall count)
+                                           :from refs
+                                           :where (= file $s1)]
+                                          (abs-path "foo.org"))) :to-be 0)
+              (expect (caar (org-roam-sql [:select (funcall count)
+                                           :from file-links
+                                           :where (= file-from $s1)]
+                                          (abs-path "foo.org"))) :to-be 0))
+
+          (it "delete web_ref"
+              (expect (org-roam-sql [:select * :from refs])
+                      :to-have-same-items-as
+                      (list (list "https://google.com/" (abs-path "web_ref.org"))))
+              (delete-file (abs-path "web_ref.org"))
+              (expect (org-roam-sql [:select * :from refs])
+                      :to-have-same-items-as
+                      (list))))