Skip to content

Optimise ledger parsing with caching #270

@wuzzapcom

Description

@wuzzapcom

I have a personal Beancount ledger with transactions spanning three years, so the total number of transactions to parse is around 10,000.
On my Raspberry Pi, even a basic query may take tens of seconds when using bean-query. My assumption is that parsing takes a significant amount of time and CPU resources, and that it would be worth optimizing this by using in-memory caching between different bean-query calls.

My idea is the following:

  • bean-query gets a new option --daemon

    • in this mode, the tool spawns a background process tied to a given ledger file

      • when any of the files change, the tool reparses them, possibly with some delay
    • the daemon keeps parsed transactions in memory and serves them to bean-query on request

  • when bean-query is called without --daemon, it looks for a running background daemon and requests the parsed transactions from it

This mode could be useful for server-side use cases such as Fava. I haven’t calculated the RAM usage, but even if it consumes 10 MB of RAM for a 1 MB ledger, that seems like a reasonable trade-off.

So my questions are:

  1. What do you think of this idea in general?
  2. Would you consider implementing it?
  3. If not, would you accept this feature as a pull request and merge it into master?

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions