Every file has a title when saved into the database, so this function is
redundant in most cases (e.g. in org-roam-buffer, where backlinks are
fetched from the database).
* (feat): add `org-roam-db-update-method`
Add `org-roam-db-update-method`, which can be one of two choices:
1. `immediate`: the cache is updated upon file changes
2. `idle-timer`: Marks the org-roam database as dirty. If Emacs idles
for some seconds, the Org-roam cache is updated. This is the default,
and current behaviour.
The idle method makes for a smoother editing experience, but some
inconsistencies can be faced.
* org-roam-update-db-idle-seconds -> org-roam-db-update-idle-seconds
...about undefined functions or unused lexical variables.
Undefined functions: for example, s.el and org-element should be
`require`'d when their functions are used.
Unused lexical variables: if Org isn't loaded yet, dynamic variables
defined in org.el would be treated as lexical and byte-comp would emit
this warning. This is especially important in the future as
native-comp / gccemacs will optimize away unused lexical variables,
and we cannot rely on Org having been implicitly loaded before our
modules are compiled.
Explicitly stating in our modules that the variables are dynamic
prevents that.
Instead of maintaining a file queue to process for updating the Org-roam
database, we instead simply call `org-roam-db-build-cache` to rebuild
the db. `org-roam-db-build-cache` feels Fast Enough(TM), and basically
runs instantly if no files are modified. This greatly simplifies the
code, basically allowing to maintaining a single code path for db
operations.
This PR should also address the slowness wrt to ID links. Org-roam now
builds the headline information first, so there is no need to read the
file to check if the headline is there. The downside to this is that
this only works for IDs in Org-roam files.
* (fix): org-roam-db-build-cache: fix order of processing
Org-roam used to perform the removal of deleted files towards the end.
This can cause some issues with db rebuilds. Consider this scenario:
1. Create a file `foo.org` with `id:abc`
2. Run `org-roam-db-build-cache`
3. Delete `foo.org`
4. Create a file `bar.org` with `id:abc`
5. Run `org-roam-db-build-cache`
Here Org-roam will complain that the id `abc` is a duplicate, and error
out, because the data for `foo.org` has not yet been cleared from the
database. This PR reorders the db creation steps the following way:
1. Figure out which files no longer exist, and which files are modified
2. Clear the database for these files, leaving only files that are
unmodified in the database
3. Insert new data from modified files into the database
Move the default location of `org-roam.db` to the user's Emacs
directory. This is a more sensible default: those who sync their
Org files would not have the database synced over as well.
Additionally cache IDs at outline-level 0, now that property drawers are supported in Org 9.4.
Update org-roam--format-link to prefer ID links wherever possible. That is, when a file has an ID, use an id link instead of file link.
Previously we had overwritten the Org's store link function for files,
to create IDs if the point was under the headline. This has several
issues:
1. It becomes impossible to store a link to the file using `org-store-link`
2. This override is global across Org, although we had a guard in
`org-roam-store-link`
3. IDs are created during an org-capture, because org-capture calls `org-store-link`
This is unnecessary. Org provides `org-id-store-link` instead for this
purpose. Instead, we advice `org-id-new`, to queue the file for a DB
update if an ID was created.
`org-mode` builds the org-agenda-files menu entry for every file
visited. This functionality isn't required and useful when building
the cache in temporary org buffers.
`find-file-noselect` is affected by the user's init.el, and there is
little control over what is being activated. For database updates, read
access is all we need, so we use `with-temp-file`.
file-truename calls alone accounts for over 20% of CPU samples in org-roam-db-build-cache. It's expensive and unnecessary. To get an absolute path, expand-file-name alone is enough.
Co-authored-by: Jethro Kuan <jethrokuan95@gmail.com>
Adds `org-roam-title-change-hook`. This now contains two functions:
1. `org-roam--update-links-on-title-change`: updates the link
description of all files that link to the current file.
2. `org-roam--update-file-name-on-title-change`: updates the current
buffer's filename to reflect the updated title.
When using gpg encrypted files it might happen (intended) that on a
certain machine there is no key to decrypt that file. Currently an
error of type 'file-error' will be raised and the cache building
process will be aborted. So in a certain sense org-roam will not
be functional in that case. Furthermore, when the the cache building
process is run from the emacs initialisation it will come to a halt as
well and the user will be left with a halfworking emacs instance.
This patch changes the behaviour to just skipping over offending files
while removing them from the db at the same time. Here an offending
file is any file that cannot be read for indexing. As it stands this
case only works reliably for the case of missing gpg keys.
An error buffer will still show up and be prominently displayed,
besides a log to the message buffer.
Implementation note: This io error will only be caught during the
first part of the cache rebuilding ("files and headlines"). If such
an error would occur during the second part ("rebuild the rest")
another (more severe) cause might be the problem and the user better
be informed the hard way (i.e. the same behaviour as before).
Before this patch all hash-sums were computed over the files or
buffers content. For encrypted files this means that we first have
decrypt the file before we can compute the hash-sum. So when the
cache get's updated all gpg files need to be decrypted which is a very
expensive operation and nearly defeats the purpose of having a cache
in the first place (for gpg files).
This changes the computation of hash-sums for gpg encrypted files.
Instead of the content the raw files on disk will be read.
This shouldn't interfere with the current use of hashes.
There is one ugly (but otherwise inconsequential) ward, though.
For open buffers of to be gpg encrypted files we need to compute the
hash sum over the contents as well. This is because there is
no (easy) way to get the encrypted version. The consequence is that
that buffers file will be rehashed again (then using the bytes on
disk). But all other non changed gpg files will only be hashed once,
as desired.
Co-authored-by: Jethro Kuan <jethrokuan95@gmail.com>
7f56df7f4d introduced the delta symbol in the
db-rebuild message to make it clearer the the db was updated as opposed to
being completely rebuilt.
However, the unicode symbol used, U+1D6AB, which is the mathematical, bold
variant of the Greek letter is not present in many fonts. Switching to
U+0394, the base Greek letter, is better for compatibility.
Instead of storing titles as a list within in the Org-roam cache, e.g.
file, ("title1" "title2"). We normalize it and store it as:
file, "title1"
file, "title2"
The high value has been reported to cause significant slowdowns, so we
reset the default, and add documentation for its customization instead.
Ref #834
We use sqlite transactions to commit changes into the database, rather
than storing all the data in a list before running one big insert.
Hopefully this gives a noticeable perf boost.
We also add `org-roam-db-gc-threshold`, which shaves time by deferring the garbage collection to the end.
Achieve feature parity between links to files and links to headlines.
Before, we used the `file:foo::*bar` format to link to the headline `bar` in file `foo`, but this was prone to breakage upon renaming the file or modifying the headline. This is not the case anymore. Now, we use `org-id` to create IDs for those headlines, which are then stored in our database to compute the relationships and jump around. Note that this will work even if you’re not using `org-id` in your global configuration for Org-mode.
Co-authored-by: Jethro Kuan <jethrokuan95@gmail.com>
Org-roam now skips over bad properties and throws a warning for the
given file that contains a malformed property. This allows most of the
database rebuild to complete, and for the user to fix the offending
file.
Fixes#728.
