Revert from emacsql-sqlite3 to emacsql-sqlite for foreign key support.
This makes it slightly harder to get Org-roam up and running for
Windows, but not impossible, but foreign keys makes database operations
faster and keeps the code clean.
Also, fix schemata for foreign keys on various tables, and add indices
where joins are common (org-roam-db--table-indices).
In my case of testing with 8253 zettels (available on request if
anyone wants to test with them), this improves cleanup speed by 6x
and update speed by 2x.
Improvements will be smaller for smaller batches as this really just
removes the transaction overhead.
The rename file advice is passed relative file names: e.g. "foo.org" ->
"bar.org". This was not accounted for, and paths in the Org-roam
database are supposed to be absolute paths. This caused the storing of
relative paths in the Org-roam database, which were then never purged.
Fixes#1304
Finding an Org-roam file that does not exist yet triggers the find-file
hook, resulting in an Org-roam buffer update, attempting to show the
file. However, if the file does not yet exist, attempts to show the file
will not work, since the database does not contain the relevant
information yet.
This bug has always been present, but was only recently made visible by
the code cleanups in #1284.
Now, we ensure that the file has been processed by the database before
attempting to re-display the buffer.
Fixes#1297
Every file has a title when saved into the database, so this function is
redundant in most cases (e.g. in org-roam-buffer, where backlinks are
fetched from the database).
* (feat): add `org-roam-db-update-method`
Add `org-roam-db-update-method`, which can be one of two choices:
1. `immediate`: the cache is updated upon file changes
2. `idle-timer`: Marks the org-roam database as dirty. If Emacs idles
for some seconds, the Org-roam cache is updated. This is the default,
and current behaviour.
The idle method makes for a smoother editing experience, but some
inconsistencies can be faced.
* org-roam-update-db-idle-seconds -> org-roam-db-update-idle-seconds
...about undefined functions or unused lexical variables.
Undefined functions: for example, s.el and org-element should be
`require`'d when their functions are used.
Unused lexical variables: if Org isn't loaded yet, dynamic variables
defined in org.el would be treated as lexical and byte-comp would emit
this warning. This is especially important in the future as
native-comp / gccemacs will optimize away unused lexical variables,
and we cannot rely on Org having been implicitly loaded before our
modules are compiled.
Explicitly stating in our modules that the variables are dynamic
prevents that.
Instead of maintaining a file queue to process for updating the Org-roam
database, we instead simply call `org-roam-db-build-cache` to rebuild
the db. `org-roam-db-build-cache` feels Fast Enough(TM), and basically
runs instantly if no files are modified. This greatly simplifies the
code, basically allowing to maintaining a single code path for db
operations.
This PR should also address the slowness wrt to ID links. Org-roam now
builds the headline information first, so there is no need to read the
file to check if the headline is there. The downside to this is that
this only works for IDs in Org-roam files.
* (fix): org-roam-db-build-cache: fix order of processing
Org-roam used to perform the removal of deleted files towards the end.
This can cause some issues with db rebuilds. Consider this scenario:
1. Create a file `foo.org` with `id:abc`
2. Run `org-roam-db-build-cache`
3. Delete `foo.org`
4. Create a file `bar.org` with `id:abc`
5. Run `org-roam-db-build-cache`
Here Org-roam will complain that the id `abc` is a duplicate, and error
out, because the data for `foo.org` has not yet been cleared from the
database. This PR reorders the db creation steps the following way:
1. Figure out which files no longer exist, and which files are modified
2. Clear the database for these files, leaving only files that are
unmodified in the database
3. Insert new data from modified files into the database
Move the default location of `org-roam.db` to the user's Emacs
directory. This is a more sensible default: those who sync their
Org files would not have the database synced over as well.
Additionally cache IDs at outline-level 0, now that property drawers are supported in Org 9.4.
Update org-roam--format-link to prefer ID links wherever possible. That is, when a file has an ID, use an id link instead of file link.
Previously we had overwritten the Org's store link function for files,
to create IDs if the point was under the headline. This has several
issues:
1. It becomes impossible to store a link to the file using `org-store-link`
2. This override is global across Org, although we had a guard in
`org-roam-store-link`
3. IDs are created during an org-capture, because org-capture calls `org-store-link`
This is unnecessary. Org provides `org-id-store-link` instead for this
purpose. Instead, we advice `org-id-new`, to queue the file for a DB
update if an ID was created.
`org-mode` builds the org-agenda-files menu entry for every file
visited. This functionality isn't required and useful when building
the cache in temporary org buffers.
