mirror of
https://github.com/org-roam/org-roam
synced 2025-08-01 12:17:21 -05:00
(feat) core: support new org-mode citations (#1806)
Support caching the new Org 9.5 citations. Because citations now has first-class support, and are treated differently from links, they are now cached in their own table. Org-ref citations, instead of being stored in the links table, are now stored in the citations table instead. To use a citation as a ROAM_REF, use the `@citeKey` syntax
This commit is contained in:
Jethro Kuan
GitHub
@ -2,6 +2,8 @@
|
||||
|
||||
## TBD
|
||||
### Added
|
||||
- [#1806](https://github.com/org-roam/org-roam/pull/1806) db: support caching and usage of Org 9.5's in-built citations
|
||||
|
||||
### Removed
|
||||
### Changed
|
||||
- [#1795](https://github.com/org-roam/org-roam/pull/1795) buffer: optimized reflinks fetch
|
||||
|
||||
@ -646,6 +646,49 @@ Org-roam also provides some functions to add or remove refs.
|
||||
|
||||
Remove a ref from the node at point.
|
||||
|
||||
* Citations
|
||||
|
||||
Since version 9.5, Org has first-class support for citations. Org-roam supports
|
||||
the caching of both these in-built citations (of form ~[cite:@key]~) and [[https://github.com/jkitchin/org-ref][org-ref]]
|
||||
citations (of form cite:key).
|
||||
|
||||
Org-roam attempts to load both the ~org-ref~ and ~org-cite~ package when
|
||||
indexing files, so no further setup from the user is required for citation
|
||||
support.
|
||||
|
||||
** Using the Cached Information
|
||||
|
||||
It is common to use take reference notes for academic papers. To designate the
|
||||
node to be the canonical node for the academic paper, we can use its unique
|
||||
citation key:
|
||||
|
||||
#+begin_src org
|
||||
,* Probabilistic Robotics
|
||||
:PROPERTIES:
|
||||
:ID: 51b7b82c-bbb4-4822-875a-ed548cffda10
|
||||
:ROAM_REFS: @thrun2005probabilistic
|
||||
:END:
|
||||
#+end_src
|
||||
|
||||
for ~org-cite~, or:
|
||||
|
||||
#+begin_src org
|
||||
,* Probabilistic Robotics
|
||||
:PROPERTIES:
|
||||
:ID: 51b7b82c-bbb4-4822-875a-ed548cffda10
|
||||
:ROAM_REFS: cite:thrun2005probabilistic
|
||||
:END:
|
||||
#+end_src
|
||||
|
||||
for ~org-ref~.
|
||||
|
||||
When another node has a citation for that key, we can see it using the
|
||||
~Reflinks~ section of the Org-roam buffer.
|
||||
|
||||
Extension developers may be interested in retrieving the citations within their
|
||||
notes. This information can be found within the ~citation~ table of the Org-roam
|
||||
database.
|
||||
|
||||
* Completion
|
||||
|
||||
Completions for Org-roam are provided via ~completion-at-point~. Org-roam
|
||||
|
||||
@ -70,6 +70,7 @@ General Public License for more details.
|
||||
* Customizing Node Caching::
|
||||
* The Org-roam Buffer::
|
||||
* Node Properties::
|
||||
* Citations::
|
||||
* Completion::
|
||||
* Encryption::
|
||||
* Org-roam Protocol::
|
||||
@ -86,6 +87,7 @@ General Public License for more details.
|
||||
* Command Index::
|
||||
* Function Index::
|
||||
* Variable Index::
|
||||
* Bibliography: Bibliography (1).
|
||||
|
||||
@detailmenu
|
||||
--- The Detailed Node Listing ---
|
||||
@ -126,6 +128,10 @@ Node Properties
|
||||
* Tags::
|
||||
* Refs::
|
||||
|
||||
Citations
|
||||
|
||||
* Using the Cached Information::
|
||||
|
||||
Completion
|
||||
|
||||
* Completing within Link Brackets::
|
||||
@ -250,7 +256,7 @@ available to Emacs.
|
||||
|
||||
Org-roam is a tool that will appear unfriendly to anyone unfamiliar with Emacs
|
||||
and Org-mode, but it is also extremely powerful to those willing to put effort
|
||||
inn mastering the intricacies. Org-roam stands on the shoulders of giants. Emacs
|
||||
in mastering the intricacies. Org-roam stands on the shoulders of giants. Emacs
|
||||
was first created in 1976, and remains the tool of choice for many for editing
|
||||
text and designing textual interfaces. The malleability of Emacs allowed the
|
||||
creation of Org-mode, an all-purpose plain-text system for maintaining TODO
|
||||
@ -637,8 +643,8 @@ The @code{file-truename} function is only necessary when you use symbolic links
|
||||
inside @code{org-roam-directory}: Org-roam does not resolve symbolic links.
|
||||
|
||||
Next, we setup Org-roam to run functions on file changes to maintain cache
|
||||
consistency. This is achieved by running @code{M-x org-roam-db-autosync-mode~}.
|
||||
To ensure that Org-roam is available on startup, place this in your Emacs
|
||||
consistency. This is achieved by running @code{M-x org-roam-db-autosync-mode}. To
|
||||
ensure that Org-roam is available on startup, place this in your Emacs
|
||||
configuration:
|
||||
|
||||
@lisp
|
||||
@ -995,6 +1001,55 @@ ref to add.
|
||||
Remove a ref from the node at point.
|
||||
@end defun
|
||||
|
||||
@node Citations
|
||||
@chapter Citations
|
||||
|
||||
Since version 9.5, Org has first-class support for citations. Org-roam supports
|
||||
the caching of both these in-built citations (of form @code{[cite:@@key]}) and @uref{https://github.com/jkitchin/org-ref, org-ref}
|
||||
citations (of form (NO@math{_ITEM}@math{_DATA}:key)).
|
||||
|
||||
Org-roam attempts to load both the @code{org-ref} and @code{org-cite} package when
|
||||
indexing files, so no further setup from the user is required for citation
|
||||
support.
|
||||
|
||||
@menu
|
||||
* Using the Cached Information::
|
||||
@end menu
|
||||
|
||||
@node Using the Cached Information
|
||||
@section Using the Cached Information
|
||||
|
||||
It is common to use take reference notes for academic papers. To designate the
|
||||
node to be the canonical node for the academic paper, we can use its unique
|
||||
citation key:
|
||||
|
||||
@example
|
||||
* Probabilistic Robotics
|
||||
:PROPERTIES:
|
||||
:ID: 51b7b82c-bbb4-4822-875a-ed548cffda10
|
||||
:ROAM_REFS: @@thrun2005probabilistic
|
||||
:END:
|
||||
@end example
|
||||
|
||||
for @code{org-cite}, or:
|
||||
|
||||
@example
|
||||
* Probabilistic Robotics
|
||||
:PROPERTIES:
|
||||
:ID: 51b7b82c-bbb4-4822-875a-ed548cffda10
|
||||
:ROAM_REFS: cite:thrun2005probabilistic
|
||||
:END:
|
||||
@end example
|
||||
|
||||
for @code{org-ref}.
|
||||
|
||||
When another node has a citation for that key, we can see it using the
|
||||
@code{Reflinks} section of the Org-roam buffer.
|
||||
|
||||
Extension developers may be interested in retrieving the citations within their
|
||||
notes. This information can be found within the @code{citation} table of the Org-roam
|
||||
database.
|
||||
|
||||
@node Completion
|
||||
@chapter Completion
|
||||
|
||||
@ -1331,12 +1386,12 @@ here.
|
||||
This template means don't insert any content, but place the cursor here.
|
||||
|
||||
@item
|
||||
@code{:target} is a compulsory specification in the Org-roam capture
|
||||
template. The first element of the list indicates the type of the
|
||||
target, the second element indicates the location of the captured
|
||||
node, and the rest of the elements indicate prefilled template that
|
||||
will be inserted and the position of the point will be adjusted for.
|
||||
The latter behavior various from type to type of the capture target.
|
||||
@code{:target} is a compulsory specification in the Org-roam capture template. The
|
||||
first element of the list indicates the type of the target, the second
|
||||
element indicates the location of the captured node, and the rest of the
|
||||
elements indicate prefilled template that will be inserted and the position
|
||||
of the point will be adjusted for. The latter behavior various from type to
|
||||
type of the capture target.
|
||||
|
||||
@item
|
||||
@code{:unnarrowed t} tells org-capture to show the contents for the whole file,
|
||||
@ -2121,5 +2176,10 @@ When GOTO is non-nil, go the note without creating an entry."
|
||||
|
||||
@printindex vr
|
||||
|
||||
@node Bibliography (1)
|
||||
@chapter Bibliography
|
||||
|
||||
NO@math{_ITEM}@math{_DATA}:key
|
||||
|
||||
Emacs 28.0.50 (Org mode 9.5)
|
||||
@bye
|
||||
|
||||
115
org-roam-db.el
115
org-roam-db.el
@ -79,7 +79,7 @@ slow."
|
||||
:group 'org-roam)
|
||||
|
||||
;;; Variables
|
||||
(defconst org-roam-db-version 16)
|
||||
(defconst org-roam-db-version 17)
|
||||
|
||||
;; TODO Rename this
|
||||
(defconst org-roam--sqlite-available-p
|
||||
@ -168,6 +168,13 @@ The query is expected to be able to fail, in this situation, run HANDLER."
|
||||
alias]
|
||||
(:foreign-key [node-id] :references nodes [id] :on-delete :cascade)))
|
||||
|
||||
(citations
|
||||
([(node-id :not-null)
|
||||
(cite-key :not-null)
|
||||
(pos :not-null)
|
||||
properties]
|
||||
(:foreign-key [node-id] :references nodes [id] :on-delete :cascade)))
|
||||
|
||||
(refs
|
||||
([(node-id :not-null)
|
||||
(ref :not-null)
|
||||
@ -284,14 +291,23 @@ If UPDATE-P is non-nil, first remove the file in the database."
|
||||
(dolist (fn fns)
|
||||
(funcall fn)))))))
|
||||
|
||||
(defun org-roam-db-map-links (fns)
|
||||
"Run FNS over all links in the current buffer."
|
||||
(defun org-roam-db-map-links (info fns)
|
||||
"Run FNS over all links in the current buffer.
|
||||
INFO is the org-element parsed buffer."
|
||||
(org-with-point-at 1
|
||||
(org-element-map (org-element-parse-buffer) 'link
|
||||
(org-element-map info 'link
|
||||
(lambda (link)
|
||||
(dolist (fn fns)
|
||||
(funcall fn link))))))
|
||||
|
||||
(defun org-roam-db-map-citations (info fns)
|
||||
"Run FNS over all citations in the current buffer.
|
||||
INFO is the org-element parsed buffer."
|
||||
(org-element-map info 'citation-reference
|
||||
(lambda (cite)
|
||||
(dolist (fn fns)
|
||||
(funcall fn cite)))))
|
||||
|
||||
(defun org-roam-db-insert-file-node ()
|
||||
"Insert the file-level node into the Org-roam cache."
|
||||
(org-with-point-at 1
|
||||
@ -309,10 +325,7 @@ If UPDATE-P is non-nil, first remove the file in the database."
|
||||
(scheduled nil)
|
||||
(deadline nil)
|
||||
(level 0)
|
||||
(aliases (org-entry-get (point) "ROAM_ALIASES"))
|
||||
(aliases (when aliases (split-string-and-unquote aliases)))
|
||||
(tags org-file-tags)
|
||||
(refs (org-entry-get (point) "ROAM_REFS"))
|
||||
(properties (org-entry-properties))
|
||||
(olp nil))
|
||||
(org-roam-db-query!
|
||||
@ -331,29 +344,8 @@ If UPDATE-P is non-nil, first remove the file in the database."
|
||||
(mapcar (lambda (tag)
|
||||
(vector id (substring-no-properties tag)))
|
||||
tags)))
|
||||
(when aliases
|
||||
(org-roam-db-query
|
||||
[:insert :into aliases
|
||||
:values $v1]
|
||||
(mapcar (lambda (alias)
|
||||
(vector id alias))
|
||||
aliases)))
|
||||
(when refs
|
||||
(setq refs (split-string-and-unquote refs))
|
||||
(let (rows)
|
||||
(dolist (ref refs)
|
||||
(if (string-match org-link-plain-re ref)
|
||||
(progn
|
||||
(push (vector id (match-string 2 ref)
|
||||
(match-string 1 ref)) rows))
|
||||
(lwarn '(org-roam) :warning
|
||||
"%s:%s\tInvalid ref %s, skipping..."
|
||||
(buffer-file-name) (point) ref)))
|
||||
(when rows
|
||||
(org-roam-db-query
|
||||
[:insert :into refs
|
||||
:values $v1]
|
||||
rows)))))))))
|
||||
(org-roam-db-insert-aliases)
|
||||
(org-roam-db-insert-refs))))))
|
||||
|
||||
(cl-defun org-roam-db-insert-node-data ()
|
||||
"Insert node data for headline at point into the Org-roam cache."
|
||||
@ -413,11 +405,13 @@ If UPDATE-P is non-nil, first remove the file in the database."
|
||||
(let (rows)
|
||||
(dolist (ref refs)
|
||||
(save-match-data
|
||||
(if (string-match org-link-plain-re ref)
|
||||
(progn
|
||||
(push (vector node-id (match-string 2 ref) (match-string 1 ref)) rows))
|
||||
(lwarn '(org-roam) :warning
|
||||
"%s:%s\tInvalid ref %s, skipping..." (buffer-file-name) (point) ref))))
|
||||
(cond ((string-equal (substring ref 0 1) "@")
|
||||
(push (vector node-id (substring ref 1) "cite") rows))
|
||||
((string-match org-link-plain-re ref)
|
||||
(push (vector node-id (match-string 2 ref) (match-string 1 ref)) rows))
|
||||
(t
|
||||
(lwarn '(org-roam) :warning
|
||||
"%s:%s\tInvalid ref %s, skipping..." (buffer-file-name) (point) ref)))))
|
||||
(when rows
|
||||
(org-roam-db-query [:insert :into refs
|
||||
:values $v1]
|
||||
@ -429,24 +423,42 @@ If UPDATE-P is non-nil, first remove the file in the database."
|
||||
(goto-char (org-element-property :begin link))
|
||||
(let ((type (org-element-property :type link))
|
||||
(path (org-element-property :path link))
|
||||
(source (org-roam-id-at-point))
|
||||
(properties (list :outline (ignore-errors
|
||||
;; This can error if link is not under any headline
|
||||
(org-get-outline-path 'with-self 'use-cache))))
|
||||
(source (org-roam-id-at-point)))
|
||||
(org-get-outline-path 'with-self 'use-cache)))))
|
||||
;; For Org-ref links, we need to split the path into the cite keys
|
||||
(when (and (boundp 'org-ref-cite-types)
|
||||
(when (and source path)
|
||||
(if (and (require 'org-ref nil 'noerror)
|
||||
(boundp 'org-ref-cite-types)
|
||||
(fboundp 'org-ref-split-and-strip-string)
|
||||
(member type org-ref-cite-types))
|
||||
(setq path (org-ref-split-and-strip-string path)))
|
||||
(unless (listp path)
|
||||
(setq path (list path)))
|
||||
(when (and source path)
|
||||
(progn
|
||||
(setq path (org-ref-split-and-strip-string path))
|
||||
(org-roam-db-query
|
||||
[:insert :into citations
|
||||
:values $v1]
|
||||
(mapcar (lambda (p) (vector source p (point) properties)) path)))
|
||||
|
||||
(org-roam-db-query
|
||||
[:insert :into links
|
||||
:values $v1]
|
||||
(vector (point) source path type properties)))))))
|
||||
|
||||
(defun org-roam-db-insert-citation (citation)
|
||||
"Insert data for CITATION at current point into the Org-roam cache."
|
||||
(save-excursion
|
||||
(goto-char (org-element-property :begin citation))
|
||||
(let ((key (org-element-property :key citation))
|
||||
(source (org-roam-id-at-point))
|
||||
(properties (list :outline (ignore-errors
|
||||
;; This can error if link is not under any headline
|
||||
(org-get-outline-path 'with-self 'use-cache)))))
|
||||
(when (and source key)
|
||||
(org-roam-db-query
|
||||
[:insert :into links
|
||||
[:insert :into citations
|
||||
:values $v1]
|
||||
(mapcar (lambda (p)
|
||||
(vector (point) source p type properties))
|
||||
path))))))
|
||||
(vector source key (point) properties))))))
|
||||
|
||||
;;;; Fetching
|
||||
(defun org-roam-db--get-current-files ()
|
||||
@ -479,7 +491,8 @@ If the file exists, update the cache with information."
|
||||
(setq file-path (or file-path (buffer-file-name (buffer-base-buffer))))
|
||||
(let ((content-hash (org-roam-db--file-hash file-path))
|
||||
(db-hash (caar (org-roam-db-query [:select hash :from files
|
||||
:where (= file $s1)] file-path))))
|
||||
:where (= file $s1)] file-path)))
|
||||
info)
|
||||
(unless (string= content-hash db-hash)
|
||||
(org-roam-with-file file-path nil
|
||||
(save-excursion
|
||||
@ -494,8 +507,14 @@ If the file exists, update the cache with information."
|
||||
#'org-roam-db-insert-tags
|
||||
#'org-roam-db-insert-refs))
|
||||
(setq org-outline-path-cache nil)
|
||||
(setq info (org-element-parse-buffer))
|
||||
(org-roam-db-map-links
|
||||
(list #'org-roam-db-insert-link)))))))
|
||||
info
|
||||
(list #'org-roam-db-insert-link))
|
||||
(when (require 'org-cite nil 'noerror)
|
||||
(org-roam-db-map-citations
|
||||
info
|
||||
(list #'org-roam-db-insert-citation))))))))
|
||||
|
||||
;;;###autoload
|
||||
(defun org-roam-db-sync (&optional force)
|
||||
|
||||
@ -546,7 +546,14 @@ Sorts by title."
|
||||
:from refs
|
||||
:left-join links
|
||||
:where (= refs:node-id $s1)
|
||||
:and (= links:dest refs:ref)]
|
||||
:and (= links:dest refs:ref)
|
||||
:union
|
||||
:select :distinct [refs:ref citations:node-id
|
||||
citations:pos citations:properties]
|
||||
:from refs
|
||||
:left-join citations
|
||||
:where (= refs:node-id $s1)
|
||||
:and (= citations:cite-key refs:ref)]
|
||||
(org-roam-node-id node)))
|
||||
links)
|
||||
(pcase-dolist (`(,ref ,source-id ,pos ,properties) refs)
|
||||
@ -566,16 +573,16 @@ Sorts by title."
|
||||
|
||||
(defun org-roam-reflinks-section (node)
|
||||
"The reflinks section for NODE."
|
||||
(when (org-roam-node-refs node)
|
||||
(let* ((reflinks (seq-sort #'org-roam-reflinks-sort (org-roam-reflinks-get node))))
|
||||
(magit-insert-section (org-roam-reflinks)
|
||||
(magit-insert-heading "Reflinks:")
|
||||
(dolist (reflink reflinks)
|
||||
(org-roam-node-insert-section
|
||||
:source-node (org-roam-reflink-source-node reflink)
|
||||
:point (org-roam-reflink-point reflink)
|
||||
:properties (org-roam-reflink-properties reflink)))
|
||||
(insert ?\n)))))
|
||||
(when-let ((refs (org-roam-node-refs node))
|
||||
(reflinks (seq-sort #'org-roam-reflinks-sort (org-roam-reflinks-get node))))
|
||||
(magit-insert-section (org-roam-reflinks)
|
||||
(magit-insert-heading "Reflinks:")
|
||||
(dolist (reflink reflinks)
|
||||
(org-roam-node-insert-section
|
||||
:source-node (org-roam-reflink-source-node reflink)
|
||||
:point (org-roam-reflink-point reflink)
|
||||
:properties (org-roam-reflink-properties reflink)))
|
||||
(insert ?\n))))
|
||||
|
||||
;;;; Grep
|
||||
(defvar org-roam-grep-map
|
||||
|
||||
Reference in New Issue
Block a user