HLedger is a perfect tool for generating financial reports. However, it lacks one important functionality: the boolean "OR" operator for combining queries. It is better to demonstrate the problem using a specific example. Let's generate an example hledger journal file with a decent number of transactions and accounts used. For this purpose we will use bean-example command from beancount:
1 2 3 4 5 |
[johndoe@ArchLinux]% pip install beancount [johndoe@ArchLinux]% bean-example --date-begin 2021-01 --date-end 2021-12 > Finances_2021.beancount [johndoe@ArchLinux]% pip install beancount2ledger [johndoe@ArchLinux]% beancount2ledger -f hledger Finances_2021.beancount > Finances_2021.journal |
This will yield an example hledger journal file with different accounts. "Expenses" account has many subaccounts:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 |
Expenses:Financial:Commissions Expenses:Financial:Fees Expenses:Food:Alcohol Expenses:Food:Coffee Expenses:Food:Groceries Expenses:Food:Restaurant Expenses:Health:Dental:Insurance Expenses:Health:Life:GroupTermLife Expenses:Health:Medical:Insurance Expenses:Health:Vision:Insurance Expenses:Home:Electricity Expenses:Home:Internet Expenses:Home:Phone Expenses:Home:Rent Expenses:Taxes:Y2021:US:CityNYC Expenses:Taxes:Y2021:US:Federal Expenses:Taxes:Y2021:US:Federal:PreTax401k Expenses:Taxes:Y2021:US:Medicare Expenses:Taxes:Y2021:US:SDI Expenses:Taxes:Y2021:US:SocSec Expenses:Taxes:Y2021:US:State Expenses:Transport:Tram Expenses:Vacation |
Let's say you want to display all transactions that fulfill the following conditions:
- are in "Liabilities:US:Chase:Slate" account (which is basically a credit card),
- are not "Expenses" except "Expenses:Food:Groceries" and "Expenses:Food:Restaurant", i.e. money spent only on these two food categories and not on other expense categories,
- were executed after 2021-11,
and feed these transactions into "hledger register 'acct:Liabilities:US:Chase:Slate'" command to display the running total. In other words, we want to see food expenditures of the given two categories and refills of credit card balance from other asset accounts. Another criterion is that all operations should be packable into a one line command without creating temporary files on the hard disk.
The query on accounts implies these logical operations:
1 |
Liabilities:US:Chase:Slate AND (not(Expenses) OR Expenses:Food:(Groceries|Restaurant)) |
However, hledger supports only AND operation and there is no support for OR operation, although this feature request is still open and might be addressed in the future.
Since we are printing the transactions which meet given conditions, their order in the resulting aggregation of the OR operation is not important, because the cumulative sum of expenses/money transfers will be calculated by the final "hledger register" command which is smart enough to sort them in chronological order.
Solution using "tee"
In order to emulate "OR" operation, we can use the trick with "tee" and BASH-specific process substitution, which will print 24 transactions that meet the criteria:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 |
[johndoe@ArchLinux]% hledger -f Finances_2021.journal -b 2021-11 print "acct:Liabilities:US:Chase:Slate"| tee >(hledger -f- print "not:acct:Expenses") | hledger -f- print "Expenses:Food:(Groceries|Restaurant)" 2021-11-07 * Chase:Slate | Paying off credit card Liabilities:US:Chase:Slate 705.99 USD Assets:US:BofA:Checking -705.99 USD ... 2021-12-28 * Cafe Modagor | Eating out with Bill Liabilities:US:Chase:Slate -53.88 USD Expenses:Food:Restaurant 53.88 USD 2021-12-28 * Corner Deli | Buying groceries Liabilities:US:Chase:Slate -76.47 USD Expenses:Food:Groceries 76.47 USD |
STDOUT of tee is piped into STDIN of hledger -f- print "acct:Expenses:Food:(Groceries|Restaurant)" and instead of file.txt process substitution >(hledger -f- print "not:acct:Expenses") is used, where temporary file descriptors are created. Although 'hledger -f- print "not:acct:Expenses"' is an another process, it's STDIN is mapped to STDIN of a temporary file and >(hledger -f- print "not:acct:Expenses") is treated as a file descriptor.
What happens if we pipe these transactions into "hledger register"? We have a problem here: transaction with date 2021-11-07 is not processed:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 |
[johndoe@ArchLinux]% hledger -f Finances_2021.journal -b 2021-11 print "acct:Liabilities:US:Chase:Slate"| tee >(hledger -f- print "not:acct:Expenses") | hledger -f- print "acct:Expenses:Food:(Groceries|Restaurant)" | hledger -f- register "Liabilities:US:Chase:Slate" -w 115,48 2021-11-07 * Chase:Slate | Paying off credit card Liabilities:US:Chase:Slate 705.99 USD Assets:US:BofA:Checking -705.99 USD 2021-11-03 Chichipotle | Eating out with Julie Liabilities:US:Chase:Slate -34.73 USD -34.73 USD 2021-11-05 China Garden | Eating out Liabilities:US:Chase:Slate -16.12 USD -50.85 USD 2021-11-08 Farmer Fresh | Buying groceries Liabilities:US:Chase:Slate -60.74 USD -111.59 USD 2021-11-09 Uncle Boons | Eating out with Julie Liabilities:US:Chase:Slate -25.33 USD -136.92 USD 2021-11-11 Jewel of Morroco | Eating out with Joe Liabilities:US:Chase:Slate -41.01 USD -177.93 USD 2021-11-15 Chichipotle | Eating out with Natasha Liabilities:US:Chase:Slate -34.97 USD -212.90 USD 2021-11-18 Uncle Boons | Eating out with Natasha Liabilities:US:Chase:Slate -20.12 USD -233.02 USD 2021-11-20 Uncle Boons | Eating out with Natasha Liabilities:US:Chase:Slate -20.64 USD -253.66 USD 2021-11-20 Good Moods Market | Buying groceries Liabilities:US:Chase:Slate -70.98 USD -324.64 USD 2021-11-24 China Garden | Eating out with Bill Liabilities:US:Chase:Slate -35.04 USD -359.68 USD 2021-11-29 Jewel of Morroco | Eating out with work buddies Liabilities:US:Chase:Slate -37.89 USD -397.57 USD 2021-12-03 China Garden | Eating out with Joe Liabilities:US:Chase:Slate -35.01 USD -432.58 USD 2021-12-06 Good Moods Market | Buying groceries Liabilities:US:Chase:Slate -35.74 USD -468.32 USD 2021-12-08 China Garden | Eating out with Natasha Liabilities:US:Chase:Slate -17.76 USD -486.08 USD 2021-12-11 Rose Flower | Eating out with Bill Liabilities:US:Chase:Slate -41.65 USD -527.73 USD 2021-12-15 Rose Flower | Eating out alone Liabilities:US:Chase:Slate -28.48 USD -556.21 USD 2021-12-17 Cafe Modagor | Eating out with Julie Liabilities:US:Chase:Slate -39.99 USD -596.20 USD 2021-12-18 Corner Deli | Buying groceries Liabilities:US:Chase:Slate -91.01 USD -687.21 USD 2021-12-20 Uncle Boons | Eating out with Joe Liabilities:US:Chase:Slate -36.08 USD -723.29 USD 2021-12-21 Goba Goba | Eating out with Joe Liabilities:US:Chase:Slate -53.08 USD -776.37 USD 2021-12-24 Jewel of Morroco | Eating out after work Liabilities:US:Chase:Slate -19.74 USD -796.11 USD 2021-12-28 Cafe Modagor | Eating out with Bill Liabilities:US:Chase:Slate -53.88 USD -849.99 USD 2021-12-28 Corner Deli | Buying groceries Liabilities:US:Chase:Slate -76.47 USD -926.46 USD |
We can fix it by combining STDOUT of two commands (two "hledger print"s for filtering account names) before the last command ("hledger register"):
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 |
[johndoe@ArchLinux]% { { hledger -f Finances_2021.journal -b 2021-11 print "acct:Liabilities:US:Chase:Slate" | tee /dev/fd/3 | hledger -f- print "acct:Expenses:Food:(Groceries|Restaurant)" >&4; } 3>&1 | hledger -f- print "not:acct:Expenses" ;} 4>&1 | hledger -f- register "acct:Liabilities:US:Chase:Slate" -w 115,48 | cat -n 1 2021-11-03 Chichipotle | Eating out with Julie Liabilities:US:Chase:Slate -34.73 USD -34.73 USD 2 2021-11-05 China Garden | Eating out Liabilities:US:Chase:Slate -16.12 USD -50.85 USD 3 2021-11-07 Chase:Slate | Paying off credit card Liabilities:US:Chase:Slate 705.99 USD 655.14 USD 4 2021-11-08 Farmer Fresh | Buying groceries Liabilities:US:Chase:Slate -60.74 USD 594.40 USD 5 2021-11-09 Uncle Boons | Eating out with Julie Liabilities:US:Chase:Slate -25.33 USD 569.07 USD 6 2021-11-11 Jewel of Morroco | Eating out with Joe Liabilities:US:Chase:Slate -41.01 USD 528.06 USD 7 2021-11-15 Chichipotle | Eating out with Natasha Liabilities:US:Chase:Slate -34.97 USD 493.09 USD 8 2021-11-18 Uncle Boons | Eating out with Natasha Liabilities:US:Chase:Slate -20.12 USD 472.97 USD 9 2021-11-20 Uncle Boons | Eating out with Natasha Liabilities:US:Chase:Slate -20.64 USD 452.33 USD 10 2021-11-20 Good Moods Market | Buying groceries Liabilities:US:Chase:Slate -70.98 USD 381.35 USD 11 2021-11-24 China Garden | Eating out with Bill Liabilities:US:Chase:Slate -35.04 USD 346.31 USD 12 2021-11-29 Jewel of Morroco | Eating out with work buddies Liabilities:US:Chase:Slate -37.89 USD 308.42 USD 13 2021-12-03 China Garden | Eating out with Joe Liabilities:US:Chase:Slate -35.01 USD 273.41 USD 14 2021-12-06 Good Moods Market | Buying groceries Liabilities:US:Chase:Slate -35.74 USD 237.67 USD 15 2021-12-08 China Garden | Eating out with Natasha Liabilities:US:Chase:Slate -17.76 USD 219.91 USD 16 2021-12-11 Rose Flower | Eating out with Bill Liabilities:US:Chase:Slate -41.65 USD 178.26 USD 17 2021-12-15 Rose Flower | Eating out alone Liabilities:US:Chase:Slate -28.48 USD 149.78 USD 18 2021-12-17 Cafe Modagor | Eating out with Julie Liabilities:US:Chase:Slate -39.99 USD 109.79 USD 19 2021-12-18 Corner Deli | Buying groceries Liabilities:US:Chase:Slate -91.01 USD 18.78 USD 20 2021-12-20 Uncle Boons | Eating out with Joe Liabilities:US:Chase:Slate -36.08 USD -17.30 USD 21 2021-12-21 Goba Goba | Eating out with Joe Liabilities:US:Chase:Slate -53.08 USD -70.38 USD 22 2021-12-24 Jewel of Morroco | Eating out after work Liabilities:US:Chase:Slate -19.74 USD -90.12 USD 23 2021-12-28 Cafe Modagor | Eating out with Bill Liabilities:US:Chase:Slate -53.88 USD -144.00 USD 24 2021-12-28 Corner Deli | Buying groceries Liabilities:US:Chase:Slate -76.47 USD -220.47 USD |
Changing the filtering at the last "hledger register" command to display "Expense:*" accounts of the same transactions:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 |
[johndoe@ArchLinux]% { { hledger -f Finances_2021.journal -b 2021-11 print "acct:Liabilities:US:Chase:Slate" | tee /dev/fd/3 | hledger -f- print "acct:Expenses:Food:(Groceries|Restaurant)" >&4; } 3>&1 | hledger -f- print "not:acct:Expenses" ;} 4>&1 | hledger -f- register "not:acct:Liabilities:US:Chase:Slate" -w 115,48 | cat -n 1 2021-11-03 Chichipotle | Eating out with Julie Expenses:Food:Restaurant 34.73 USD 34.73 USD 2 2021-11-05 China Garden | Eating out Expenses:Food:Restaurant 16.12 USD 50.85 USD 3 2021-11-07 Chase:Slate | Paying off credit card Assets:US:BofA:Checking -705.99 USD -655.14 USD 4 2021-11-08 Farmer Fresh | Buying groceries Expenses:Food:Groceries 60.74 USD -594.40 USD 5 2021-11-09 Uncle Boons | Eating out with Julie Expenses:Food:Restaurant 25.33 USD -569.07 USD 6 2021-11-11 Jewel of Morroco | Eating out with Joe Expenses:Food:Restaurant 41.01 USD -528.06 USD 7 2021-11-15 Chichipotle | Eating out with Natasha Expenses:Food:Restaurant 34.97 USD -493.09 USD 8 2021-11-18 Uncle Boons | Eating out with Natasha Expenses:Food:Restaurant 20.12 USD -472.97 USD 9 2021-11-20 Uncle Boons | Eating out with Natasha Expenses:Food:Restaurant 20.64 USD -452.33 USD 10 2021-11-20 Good Moods Market | Buying groceries Expenses:Food:Groceries 70.98 USD -381.35 USD 11 2021-11-24 China Garden | Eating out with Bill Expenses:Food:Restaurant 35.04 USD -346.31 USD 12 2021-11-29 Jewel of Morroco | Eating out with work buddies Expenses:Food:Restaurant 37.89 USD -308.42 USD 13 2021-12-03 China Garden | Eating out with Joe Expenses:Food:Restaurant 35.01 USD -273.41 USD 14 2021-12-06 Good Moods Market | Buying groceries Expenses:Food:Groceries 35.74 USD -237.67 USD 15 2021-12-08 China Garden | Eating out with Natasha Expenses:Food:Restaurant 17.76 USD -219.91 USD 16 2021-12-11 Rose Flower | Eating out with Bill Expenses:Food:Restaurant 41.65 USD -178.26 USD 17 2021-12-15 Rose Flower | Eating out alone Expenses:Food:Restaurant 28.48 USD -149.78 USD 18 2021-12-17 Cafe Modagor | Eating out with Julie Expenses:Food:Restaurant 39.99 USD -109.79 USD 19 2021-12-18 Corner Deli | Buying groceries Expenses:Food:Groceries 91.01 USD -18.78 USD 20 2021-12-20 Uncle Boons | Eating out with Joe Expenses:Food:Restaurant 36.08 USD 17.30 USD 21 2021-12-21 Goba Goba | Eating out with Joe Expenses:Food:Restaurant 53.08 USD 70.38 USD 22 2021-12-24 Jewel of Morroco | Eating out after work Expenses:Food:Restaurant 19.74 USD 90.12 USD 23 2021-12-28 Cafe Modagor | Eating out with Bill Expenses:Food:Restaurant 53.88 USD 144.00 USD 24 2021-12-28 Corner Deli | Buying groceries Expenses:Food:Groceries 76.47 USD 220.47 USD |
Explanation: the output of 'hledger -f- print "acct:Expenses:Food:(Groceries|Restaurant)" ' is redirected into a new stream &4 and the output of 'hledger -f Finances_2021.journal -b 2021-11 print "acct:Liabilities:US:Chase:Slate"' is redirected into a new stream &3. These output streams are combined into STDOUT (stream &1) in these two redirections: "3>&1" and "4>&1".
Solution using "pee"
Another alternative is to use "pee" command from moreutils package:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 |
[johndoe@ArchLinux]% sudo pacman -S moreutils [johndoe@ArchLinux]% hledger -f Finances_2021.journal -b 2021-11 print "acct:Liabilities:US:Chase:Slate" | pee 'hledger -f- print "Expenses:Food:(Groceries|Restaurant)"' 'hledger -f- print "not:acct:Expenses"' | hledger -f- register "acct:Liabilities:US:Chase:Slate" -w 115,48 | cat -n 1 2021-11-03 Chichipotle | Eating out with Julie Liabilities:US:Chase:Slate -34.73 USD -34.73 USD 2 2021-11-05 China Garden | Eating out Liabilities:US:Chase:Slate -16.12 USD -50.85 USD 3 2021-11-07 Chase:Slate | Paying off credit card Liabilities:US:Chase:Slate 705.99 USD 655.14 USD 4 2021-11-08 Farmer Fresh | Buying groceries Liabilities:US:Chase:Slate -60.74 USD 594.40 USD 5 2021-11-09 Uncle Boons | Eating out with Julie Liabilities:US:Chase:Slate -25.33 USD 569.07 USD 6 2021-11-11 Jewel of Morroco | Eating out with Joe Liabilities:US:Chase:Slate -41.01 USD 528.06 USD 7 2021-11-15 Chichipotle | Eating out with Natasha Liabilities:US:Chase:Slate -34.97 USD 493.09 USD 8 2021-11-18 Uncle Boons | Eating out with Natasha Liabilities:US:Chase:Slate -20.12 USD 472.97 USD 9 2021-11-20 Uncle Boons | Eating out with Natasha Liabilities:US:Chase:Slate -20.64 USD 452.33 USD 10 2021-11-20 Good Moods Market | Buying groceries Liabilities:US:Chase:Slate -70.98 USD 381.35 USD 11 2021-11-24 China Garden | Eating out with Bill Liabilities:US:Chase:Slate -35.04 USD 346.31 USD 12 2021-11-29 Jewel of Morroco | Eating out with work buddies Liabilities:US:Chase:Slate -37.89 USD 308.42 USD 13 2021-12-03 China Garden | Eating out with Joe Liabilities:US:Chase:Slate -35.01 USD 273.41 USD 14 2021-12-06 Good Moods Market | Buying groceries Liabilities:US:Chase:Slate -35.74 USD 237.67 USD 15 2021-12-08 China Garden | Eating out with Natasha Liabilities:US:Chase:Slate -17.76 USD 219.91 USD 16 2021-12-11 Rose Flower | Eating out with Bill Liabilities:US:Chase:Slate -41.65 USD 178.26 USD 17 2021-12-15 Rose Flower | Eating out alone Liabilities:US:Chase:Slate -28.48 USD 149.78 USD 18 2021-12-17 Cafe Modagor | Eating out with Julie Liabilities:US:Chase:Slate -39.99 USD 109.79 USD 19 2021-12-18 Corner Deli | Buying groceries Liabilities:US:Chase:Slate -91.01 USD 18.78 USD 20 2021-12-20 Uncle Boons | Eating out with Joe Liabilities:US:Chase:Slate -36.08 USD -17.30 USD 21 2021-12-21 Goba Goba | Eating out with Joe Liabilities:US:Chase:Slate -53.08 USD -70.38 USD 22 2021-12-24 Jewel of Morroco | Eating out after work Liabilities:US:Chase:Slate -19.74 USD -90.12 USD 23 2021-12-28 Cafe Modagor | Eating out with Bill Liabilities:US:Chase:Slate -53.88 USD -144.00 USD 24 2021-12-28 Corner Deli | Buying groceries Liabilities:US:Chase:Slate -76.47 USD -220.47 USD |
it acts like "tee", but pipes STDOUT directly into multiple commands. The STDOUT of these multiple commands is combined before entering the next pipe.
Solution attempt using regular expressions
Yet another way to emulate logical OR for filtering accounts with hierarchical structure is to use regular expressions. We want to exclude the "Expenses" account (so negation using hledger's filtering specifier "not:acct:" for a query) and all its children except "Expenses:Food:Groceries" and "Expenses:Food:Restaurant". The first thing which comes to mind is to use negative lookahead, i.e. something like "not:acct:Expenses:(?!Food:Groceries)", but as it turns out, lookaheads are not supported by hledger's regular expressions engine.
How about using matching a single character that is not contained within the brackets? The expression "not:acct:Expenses:[^F]" should filter out all expense accounts except ones whose names are starting with "Expenses:F". The problem is that "Expenses:Food" is not the only account matching this expression, there is "Expenses:Finances" as well:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 |
[johndoe@ArchLinux]% hledger -f Finances_2021.journal -b 2021-11 print "not:acct:Expenses:[^F]" | hledger -f- register "not:acct:Liabilities:US:Chase:Slate" "acct:Expenses" -w 120,48 2021-11-03 Chichipotle | Eating out with Julie Expenses:Food:Restaurant 34.73 USD 34.73 USD 2021-11-04 BANK FEES | Monthly bank fee Expenses:Financial:Fees 4.00 USD 38.73 USD 2021-11-05 China Garden | Eating out Expenses:Food:Restaurant 16.12 USD 54.85 USD 2021-11-08 Farmer Fresh | Buying groceries Expenses:Food:Groceries 60.74 USD 115.59 USD 2021-11-09 Uncle Boons | Eating out with Julie Expenses:Food:Restaurant 25.33 USD 140.92 USD 2021-11-11 Jewel of Morroco | Eating out with Joe Expenses:Food:Restaurant 41.01 USD 181.93 USD 2021-11-15 Chichipotle | Eating out with Natasha Expenses:Food:Restaurant 34.97 USD 216.90 USD 2021-11-18 Uncle Boons | Eating out with Natasha Expenses:Food:Restaurant 20.12 USD 237.02 USD 2021-11-20 Uncle Boons | Eating out with Natasha Expenses:Food:Restaurant 20.64 USD 257.66 USD 2021-11-20 Good Moods Market | Buying groceries Expenses:Food:Groceries 70.98 USD 328.64 USD 2021-11-24 China Garden | Eating out with Bill Expenses:Food:Restaurant 35.04 USD 363.68 USD 2021-11-29 Jewel of Morroco | Eating out with work buddies Expenses:Food:Restaurant 37.89 USD 401.57 USD 2021-12-03 China Garden | Eating out with Joe Expenses:Food:Restaurant 35.01 USD 436.58 USD 2021-12-03 Sell shares of VHT Expenses:Financial:Commissions 8.95 USD 445.53 USD 2021-12-04 BANK FEES | Monthly bank fee Expenses:Financial:Fees 4.00 USD 449.53 USD 2021-12-05 Buy shares of GLD Expenses:Financial:Commissions 8.95 USD 458.48 USD 2021-12-06 Good Moods Market | Buying groceries Expenses:Food:Groceries 35.74 USD 494.22 USD 2021-12-08 China Garden | Eating out with Natasha Expenses:Food:Restaurant 17.76 USD 511.98 USD 2021-12-11 Rose Flower | Eating out with Bill Expenses:Food:Restaurant 41.65 USD 553.63 USD 2021-12-15 Rose Flower | Eating out alone Expenses:Food:Restaurant 28.48 USD 582.11 USD 2021-12-17 Cafe Modagor | Eating out with Julie Expenses:Food:Restaurant 39.99 USD 622.10 USD 2021-12-18 Corner Deli | Buying groceries Expenses:Food:Groceries 91.01 USD 713.11 USD 2021-12-19 Sell shares of VHT Expenses:Financial:Commissions 8.95 USD 722.06 USD 2021-12-20 Uncle Boons | Eating out with Joe Expenses:Food:Restaurant 36.08 USD 758.14 USD 2021-12-21 Goba Goba | Eating out with Joe Expenses:Food:Restaurant 53.08 USD 811.22 USD 2021-12-24 Jewel of Morroco | Eating out after work Expenses:Food:Restaurant 19.74 USD 830.96 USD 2021-12-28 Cafe Modagor | Eating out with Bill Expenses:Food:Restaurant 53.88 USD 884.84 USD 2021-12-28 Corner Deli | Buying groceries Expenses:Food:Groceries 76.47 USD 961.31 USD |
The workaround has been discussed in a question on Stackoverflow. We need to use a capturing group with an "OR" statement combining at least two items:
- Matching a single character that is not contained within the brackets (so the first letter of the account name in the brackets): "[^F]",
- Similarly to the previous item but for the second letter of the account name, with condition that it is preceded by the first letter, i.e. "F[^o]".
These two items should extended by the items corresponding to the 3rd, 4th and following letters of the account name, as it is done in the chain rule for the joint probability distribution: "not:acct:Expenses:([^F]|F[^o]|Fo[^o]Foo[^d])"
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 |
[johndoe@ArchLinux]% ledger -f Finances_2021.journal -b 2021-11 print "not:acct:Expenses:([^F]|F[^o])" | hledger -f- register "not:acct:Liabilities:US:Chase:Slate" "acct:Expenses" -w 120,48 2021-11-03 Chichipotle | Eating out with Julie Expenses:Food:Restaurant 34.73 USD 34.73 USD 2021-11-05 China Garden | Eating out Expenses:Food:Restaurant 16.12 USD 50.85 USD 2021-11-08 Farmer Fresh | Buying groceries Expenses:Food:Groceries 60.74 USD 111.59 USD 2021-11-09 Uncle Boons | Eating out with Julie Expenses:Food:Restaurant 25.33 USD 136.92 USD 2021-11-11 Jewel of Morroco | Eating out with Joe Expenses:Food:Restaurant 41.01 USD 177.93 USD 2021-11-15 Chichipotle | Eating out with Natasha Expenses:Food:Restaurant 34.97 USD 212.90 USD 2021-11-18 Uncle Boons | Eating out with Natasha Expenses:Food:Restaurant 20.12 USD 233.02 USD 2021-11-20 Uncle Boons | Eating out with Natasha Expenses:Food:Restaurant 20.64 USD 253.66 USD 2021-11-20 Good Moods Market | Buying groceries Expenses:Food:Groceries 70.98 USD 324.64 USD 2021-11-24 China Garden | Eating out with Bill Expenses:Food:Restaurant 35.04 USD 359.68 USD 2021-11-29 Jewel of Morroco | Eating out with work buddies Expenses:Food:Restaurant 37.89 USD 397.57 USD 2021-12-03 China Garden | Eating out with Joe Expenses:Food:Restaurant 35.01 USD 432.58 USD 2021-12-06 Good Moods Market | Buying groceries Expenses:Food:Groceries 35.74 USD 468.32 USD 2021-12-08 China Garden | Eating out with Natasha Expenses:Food:Restaurant 17.76 USD 486.08 USD 2021-12-11 Rose Flower | Eating out with Bill Expenses:Food:Restaurant 41.65 USD 527.73 USD 2021-12-15 Rose Flower | Eating out alone Expenses:Food:Restaurant 28.48 USD 556.21 USD 2021-12-17 Cafe Modagor | Eating out with Julie Expenses:Food:Restaurant 39.99 USD 596.20 USD 2021-12-18 Corner Deli | Buying groceries Expenses:Food:Groceries 91.01 USD 687.21 USD 2021-12-20 Uncle Boons | Eating out with Joe Expenses:Food:Restaurant 36.08 USD 723.29 USD 2021-12-21 Goba Goba | Eating out with Joe Expenses:Food:Restaurant 53.08 USD 776.37 USD 2021-12-24 Jewel of Morroco | Eating out after work Expenses:Food:Restaurant 19.74 USD 796.11 USD 2021-12-28 Cafe Modagor | Eating out with Bill Expenses:Food:Restaurant 53.88 USD 849.99 USD 2021-12-28 Corner Deli | Buying groceries Expenses:Food:Groceries 76.47 USD 926.46 USD |
We need to be more specific to avoid matching "Expenses:Food:Coffee" and "Expenses:Food:Alcohol", so the regular expression should look like "not:acct:Expenses:([^F]|F[^o]|Fo[^o]Foo[^d]|Food:[^RG])":
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 |
[johndoe@ArchLinux]% hledger -f Finances_2021.journal -b 2021-11 print "not:acct:Expenses:([^F]|F[^o]|Fo[^o]Foo[^d]|Food:[^RG])" | hledger -f- register "not:acct:Liabilities:US:Chase:Slate" -w 120,48 2021-11-03 Chichipotle | Eating out with Julie Expenses:Food:Restaurant 34.73 USD 34.73 USD 2021-11-05 China Garden | Eating out Expenses:Food:Restaurant 16.12 USD 50.85 USD 2021-11-07 Chase:Slate | Paying off credit card Assets:US:BofA:Checking -705.99 USD -655.14 USD 2021-11-08 Farmer Fresh | Buying groceries Expenses:Food:Groceries 60.74 USD -594.40 USD 2021-11-09 Uncle Boons | Eating out with Julie Expenses:Food:Restaurant 25.33 USD -569.07 USD 2021-11-11 Jewel of Morroco | Eating out with Joe Expenses:Food:Restaurant 41.01 USD -528.06 USD 2021-11-15 Chichipotle | Eating out with Natasha Expenses:Food:Restaurant 34.97 USD -493.09 USD 2021-11-18 Uncle Boons | Eating out with Natasha Expenses:Food:Restaurant 20.12 USD -472.97 USD 2021-11-20 Uncle Boons | Eating out with Natasha Expenses:Food:Restaurant 20.64 USD -452.33 USD 2021-11-20 Good Moods Market | Buying groceries Expenses:Food:Groceries 70.98 USD -381.35 USD 2021-11-24 China Garden | Eating out with Bill Expenses:Food:Restaurant 35.04 USD -346.31 USD 2021-11-26 Transfering accumulated savings to other account Assets:US:BofA:Checking -5000.00 USD -5346.31 USD Assets:US:ETrade:Cash 5000.00 USD -346.31 USD 2021-11-29 Jewel of Morroco | Eating out with work buddies Expenses:Food:Restaurant 37.89 USD -308.42 USD 2021-12-03 China Garden | Eating out with Joe Expenses:Food:Restaurant 35.01 USD -273.41 USD 2021-12-06 Good Moods Market | Buying groceries Expenses:Food:Groceries 35.74 USD -237.67 USD 2021-12-08 China Garden | Eating out with Natasha Expenses:Food:Restaurant 17.76 USD -219.91 USD 2021-12-11 Rose Flower | Eating out with Bill Expenses:Food:Restaurant 41.65 USD -178.26 USD 2021-12-15 Rose Flower | Eating out alone Expenses:Food:Restaurant 28.48 USD -149.78 USD 2021-12-17 Cafe Modagor | Eating out with Julie Expenses:Food:Restaurant 39.99 USD -109.79 USD 2021-12-17 Dividends on portfolio Assets:US:ETrade:Cash 74.42 USD -35.37 USD Income:US:ETrade:GLD:Dividend -74.42 USD -109.79 USD 2021-12-18 Corner Deli | Buying groceries Expenses:Food:Groceries 91.01 USD -18.78 USD 2021-12-20 Uncle Boons | Eating out with Joe Expenses:Food:Restaurant 36.08 USD 17.30 USD 2021-12-21 Goba Goba | Eating out with Joe Expenses:Food:Restaurant 53.08 USD 70.38 USD 2021-12-24 Transfering accumulated savings to other account Assets:US:BofA:Checking -5500.00 USD -5429.62 USD Assets:US:ETrade:Cash 5500.00 USD 70.38 USD 2021-12-24 Jewel of Morroco | Eating out after work Expenses:Food:Restaurant 19.74 USD 90.12 USD 2021-12-28 Cafe Modagor | Eating out with Bill Expenses:Food:Restaurant 53.88 USD 144.00 USD 2021-12-28 Corner Deli | Buying groceries Expenses:Food:Groceries 76.47 USD 220.47 USD |
Although this regular expression is sufficient to solve our problem, it would also match and display accounts like "Expenses:Food:Grapes" if they existed. In order to be more specific, we need to extend it to capture only "Restaurant" and "Groceries" accounts. It is not easy, since there is an "OR" operator in the expression we want to invert: "Expenses:Food:(Groceries|Restaurant)".
The solution can be based on matching one letter at a time and combining the letters from two words into a range (square brackets): [RG][er][so][tc][ae][ur][ri][ae][ns][t$], here "Restaurant" and "Groceries" are grouped.
In essence, the condition on the preceding character should look like "<skipping the OR'less part>....|Food:[^RG]|Food:[RG][^er]|Food:[RG][er][^so])".
The full solution using only regular expressions is scary and ugly, but it works:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 |
[johndoe@ArchLinux]% hledger -f Finances_2021.journal -b 2021-11 print "acct:Liabilities:US:Chase:Slate" "not:acct:Expenses:([^F]|F[^o]|Fo[^o]|Foo[^d]|Food[^:]|Food:[^RG]|Food:[RG][^er]|Food:[RG][er][^so]|Food:[RG][er][so][^tc]|Food:[RG][er][so][tc][^ae]|Food:[RG][er][so][tc][ae][^ur]|Food:[RG][er][so][tc][ae][ur][^ri]|Food:[RG][er][so][tc][ae][ur][ri][^ae]|Food:[RG][er][so][tc][ae][ur][ri][ae][^ns]|Food:[RG][er][so][tc][ae][ur][ri][ae][ns][^t$])" | hledger -f- register "not:acct:Liabilities:US:Chase:Slate" -w 115,48 | cat -n 1 2021-11-03 Chichipotle | Eating out with Julie Expenses:Food:Restaurant 34.73 USD 34.73 USD 2 2021-11-05 China Garden | Eating out Expenses:Food:Restaurant 16.12 USD 50.85 USD 3 2021-11-07 Chase:Slate | Paying off credit card Assets:US:BofA:Checking -705.99 USD -655.14 USD 4 2021-11-08 Farmer Fresh | Buying groceries Expenses:Food:Groceries 60.74 USD -594.40 USD 5 2021-11-09 Uncle Boons | Eating out with Julie Expenses:Food:Restaurant 25.33 USD -569.07 USD 6 2021-11-11 Jewel of Morroco | Eating out with Joe Expenses:Food:Restaurant 41.01 USD -528.06 USD 7 2021-11-15 Chichipotle | Eating out with Natasha Expenses:Food:Restaurant 34.97 USD -493.09 USD 8 2021-11-18 Uncle Boons | Eating out with Natasha Expenses:Food:Restaurant 20.12 USD -472.97 USD 9 2021-11-20 Uncle Boons | Eating out with Natasha Expenses:Food:Restaurant 20.64 USD -452.33 USD 10 2021-11-20 Good Moods Market | Buying groceries Expenses:Food:Groceries 70.98 USD -381.35 USD 11 2021-11-24 China Garden | Eating out with Bill Expenses:Food:Restaurant 35.04 USD -346.31 USD 12 2021-11-29 Jewel of Morroco | Eating out with work buddies Expenses:Food:Restaurant 37.89 USD -308.42 USD 13 2021-12-03 China Garden | Eating out with Joe Expenses:Food:Restaurant 35.01 USD -273.41 USD 14 2021-12-06 Good Moods Market | Buying groceries Expenses:Food:Groceries 35.74 USD -237.67 USD 15 2021-12-08 China Garden | Eating out with Natasha Expenses:Food:Restaurant 17.76 USD -219.91 USD 16 2021-12-11 Rose Flower | Eating out with Bill Expenses:Food:Restaurant 41.65 USD -178.26 USD 17 2021-12-15 Rose Flower | Eating out alone Expenses:Food:Restaurant 28.48 USD -149.78 USD 18 2021-12-17 Cafe Modagor | Eating out with Julie Expenses:Food:Restaurant 39.99 USD -109.79 USD 19 2021-12-18 Corner Deli | Buying groceries Expenses:Food:Groceries 91.01 USD -18.78 USD 20 2021-12-20 Uncle Boons | Eating out with Joe Expenses:Food:Restaurant 36.08 USD 17.30 USD 21 2021-12-21 Goba Goba | Eating out with Joe Expenses:Food:Restaurant 53.08 USD 70.38 USD 22 2021-12-24 Jewel of Morroco | Eating out after work Expenses:Food:Restaurant 19.74 USD 90.12 USD 23 2021-12-28 Cafe Modagor | Eating out with Bill Expenses:Food:Restaurant 53.88 USD 144.00 USD 24 2021-12-28 Corner Deli | Buying groceries Expenses:Food:Groceries 76.47 USD 220.47 USD |
It will also match and display accounts formed from combinations of two letters in the sequence, e.g. "Expenses:Food:Rrscarren", "Expenses:Food:Rrscarrent", "Expenses:Food:Gescauias", "Expenses:Food:Gescauiast", "Expenses:Food:Rroceries", "Expenses:Food:Gestauran" etc. if they existed, so this approach cannot be considered as a proper solution.
Conlusion
The solution using "tee", works fine, but POSIX-compliance of file descriptors like "/dev/fd/3" is not clear. Order of execution of filtering commands is still not guaranteed, but it is not important anyways.
The solution using "pee" is the best, but requires installation of the command "pee". Order of execution of processes is still not guaranteed.
The solution attempt using regular expressions without lookaheads and pure hledger's filtering functionality is intimidating and may not be 100% correct if there are weird account names matching the constructed regular expression.
References
Keeping header of an output while grepping the rest for something else in BASH
Stackoverflow: Output order with process substitution
Wikipedia: Chain rule (probability)
Stackoverflow: Regular expression to match a line that doesn't contain a word