Require optional libraries 'org-ref and 'oc only once per function run:
org-roam-db-update-file and org-roam-db-sync will call these requires at
the top level instead of within the link mapper.
Instead check if the (autoloaded) function `org-cite-insert' is bound.
This prevents a lot of unnecessary filesystem I/O when generating the
org roam database.
The current implementation uses org-element-map to iterate through all
links in the current buffer. This function doesn't consider links
within properties drawer. This implementation does consider such
links.
Support caching the new Org 9.5 citations.
Because citations now has first-class support, and are treated
differently from links, they are now cached in their own table.
Org-ref citations, instead of being stored in the links table, are now
stored in the citations table instead.
To use a citation as a ROAM_REF, use the `@citeKey` syntax
Previously when ROAM_REFS is non-empty, but the refs are invalid, this
will cause an empty SQL expression and crash the db caching. This fixes
that scenario.
(condition-case nil
(test)
(t nil))
Is a construct that's only available starting from Emacs 27. Since
Org-roam supports Emacs 26, gotta use what's given.
Note: ignore-errors isn't exactly the same as the construct above from
Emacs 27. Compare to ignore-errors to would trap all possible signals,
while ignore-errors will only trap "error" signals. This, however,
shouldn't affect us in case of this refactoring.
Fixes#1777.
When inserting the GPG decrypted org buffer to to DB, make sure that
the hash of the encrypted file is stored.
This prevents reparsing of GPG files over and over again when
synchronizing the database with the filesystem.
Co-authored-by: Jethro Kuan <jethrokuan95@gmail.com>
Detangle the codebase and change how dependencies are resolved to
allow the package to better modularize and load itself without
introducing circular dependencies, especially when autoloads involved.
A force resync of the database does not work in Windows (Cygwin):
apply: Removing old name: Device or resource busy, /home/brschoen/.emacs.d/org-roam.db
Therefore, close the connection first and then try to remove the database file.
At the current state autoloads from non `org-roam.el` unable to start
loading of the package from their own file. See the following to learn
more about the problem:
https://github.com/org-roam/org-roam/issues/1590#issuecomment-885817825
This is no more than a workaround to bandage a more deeper underlying
problem. Until the more fundamental problem won't be fixed, all autoload
cookies from non "org-roam.el" file will need to manually point to the
main entry point of the package -- "org-roam.el" file, to load it
without causing errors.
Fixes#1590, #1620, #1600 and other similar issues.
The rename file advice is passed relative file names: e.g. "foo.org" ->
"bar.org". This was not accounted for, and paths in the Org-roam
database are supposed to be absolute paths. This caused the storing of
relative paths in the Org-roam database, which were then never purged.
Fixes#1304
Finding an Org-roam file that does not exist yet triggers the find-file
hook, resulting in an Org-roam buffer update, attempting to show the
file. However, if the file does not yet exist, attempts to show the file
will not work, since the database does not contain the relevant
information yet.
This bug has always been present, but was only recently made visible by
the code cleanups in #1284.
Now, we ensure that the file has been processed by the database before
attempting to re-display the buffer.
Fixes#1297
Every file has a title when saved into the database, so this function is
redundant in most cases (e.g. in org-roam-buffer, where backlinks are
fetched from the database).
* (feat): add `org-roam-db-update-method`
Add `org-roam-db-update-method`, which can be one of two choices:
1. `immediate`: the cache is updated upon file changes
2. `idle-timer`: Marks the org-roam database as dirty. If Emacs idles
for some seconds, the Org-roam cache is updated. This is the default,
and current behaviour.
The idle method makes for a smoother editing experience, but some
inconsistencies can be faced.
* org-roam-update-db-idle-seconds -> org-roam-db-update-idle-seconds
...about undefined functions or unused lexical variables.
Undefined functions: for example, s.el and org-element should be
`require`'d when their functions are used.
Unused lexical variables: if Org isn't loaded yet, dynamic variables
defined in org.el would be treated as lexical and byte-comp would emit
this warning. This is especially important in the future as
native-comp / gccemacs will optimize away unused lexical variables,
and we cannot rely on Org having been implicitly loaded before our
modules are compiled.
Explicitly stating in our modules that the variables are dynamic
prevents that.
Instead of maintaining a file queue to process for updating the Org-roam
database, we instead simply call `org-roam-db-build-cache` to rebuild
the db. `org-roam-db-build-cache` feels Fast Enough(TM), and basically
runs instantly if no files are modified. This greatly simplifies the
code, basically allowing to maintaining a single code path for db
operations.
This PR should also address the slowness wrt to ID links. Org-roam now
builds the headline information first, so there is no need to read the
file to check if the headline is there. The downside to this is that
this only works for IDs in Org-roam files.
* (fix): org-roam-db-build-cache: fix order of processing
Org-roam used to perform the removal of deleted files towards the end.
This can cause some issues with db rebuilds. Consider this scenario:
1. Create a file `foo.org` with `id:abc`
2. Run `org-roam-db-build-cache`
3. Delete `foo.org`
4. Create a file `bar.org` with `id:abc`
5. Run `org-roam-db-build-cache`
Here Org-roam will complain that the id `abc` is a duplicate, and error
out, because the data for `foo.org` has not yet been cleared from the
database. This PR reorders the db creation steps the following way:
1. Figure out which files no longer exist, and which files are modified
2. Clear the database for these files, leaving only files that are
unmodified in the database
3. Insert new data from modified files into the database
Move the default location of `org-roam.db` to the user's Emacs
directory. This is a more sensible default: those who sync their
Org files would not have the database synced over as well.
Additionally cache IDs at outline-level 0, now that property drawers are supported in Org 9.4.
Update org-roam--format-link to prefer ID links wherever possible. That is, when a file has an ID, use an id link instead of file link.
Previously we had overwritten the Org's store link function for files,
to create IDs if the point was under the headline. This has several
issues:
1. It becomes impossible to store a link to the file using `org-store-link`
2. This override is global across Org, although we had a guard in
`org-roam-store-link`
3. IDs are created during an org-capture, because org-capture calls `org-store-link`
This is unnecessary. Org provides `org-id-store-link` instead for this
purpose. Instead, we advice `org-id-new`, to queue the file for a DB
update if an ID was created.
`org-mode` builds the org-agenda-files menu entry for every file
visited. This functionality isn't required and useful when building
the cache in temporary org buffers.
`find-file-noselect` is affected by the user's init.el, and there is
little control over what is being activated. For database updates, read
access is all we need, so we use `with-temp-file`.
file-truename calls alone accounts for over 20% of CPU samples in org-roam-db-build-cache. It's expensive and unnecessary. To get an absolute path, expand-file-name alone is enough.
Co-authored-by: Jethro Kuan <jethrokuan95@gmail.com>
