HLedger is a perfect tool for generating financial reports. However, it lacks one important functionality: the boolean "OR" operator for combining queries. It is better to demonstrate the problem using a specific example. Let's generate an example hledger journal file with a decent number of transactions and accounts used. For this purpose we will use bean-example command from beancount:
|
1
2
3
4
5
|
[johndoe@ArchLinux]% pip install beancount
[johndoe@ArchLinux]% bean-example --date-begin 2021-01 --date-end 2021-12 > Finances_2021.beancount
[johndoe@ArchLinux]% pip install beancount2ledger
[johndoe@ArchLinux]% beancount2ledger -f hledger Finances_2021.beancount > Finances_2021.journal
|
This will yield an example hledger journal file with different accounts. "Expenses" account has many subaccounts:
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
|
Expenses:Financial:Commissions
Expenses:Financial:Fees
Expenses:Food:Alcohol
Expenses:Food:Coffee
Expenses:Food:Groceries
Expenses:Food:Restaurant
Expenses:Health:Dental:Insurance
Expenses:Health:Life:GroupTermLife
Expenses:Health:Medical:Insurance
Expenses:Health:Vision:Insurance
Expenses:Home:Electricity
Expenses:Home:Internet
Expenses:Home:Phone
Expenses:Home:Rent
Expenses:Taxes:Y2021:US:CityNYC
Expenses:Taxes:Y2021:US:Federal
Expenses:Taxes:Y2021:US:Federal:PreTax401k
Expenses:Taxes:Y2021:US:Medicare
Expenses:Taxes:Y2021:US:SDI
Expenses:Taxes:Y2021:US:SocSec
Expenses:Taxes:Y2021:US:State
Expenses:Transport:Tram
Expenses:Vacation
|
Let's say you want to display all transactions that fulfill the following conditions:
- are in "Liabilities:US:Chase:Slate" account (which is basically a credit card),
- are not "Expenses" except "Expenses:Food:Groceries" and "Expenses:Food:Restaurant", i.e. money spent only on these two food categories and not on other expense categories,
- were executed after 2021-11,
and feed these transactions into "hledger register 'acct:Liabilities:US:Chase:Slate'" command to display the running total. In other words, we want to see food expenditures of the given two categories and refills of credit card balance from other asset accounts. Another criterion is that all operations should be packable into a one line command without creating temporary files on the hard disk.
The query on accounts implies these logical operations:
|
1
|
Liabilities:US:Chase:Slate AND (not(Expenses) OR Expenses:Food:(Groceries|Restaurant))
|
However, hledger supports only AND operation and there is no support for OR operation, although this feature request is still open and might be addressed in the future.
Since we are printing the transactions which meet given conditions, their order in the resulting aggregation of the OR operation is not important, because the cumulative sum of expenses/money transfers will be calculated by the final "hledger register" command which is smart enough to sort them in chronological order.
Solution using "tee"
In order to emulate "OR" operation, we can use the trick with "tee" and BASH-specific process substitution, which will print 24 transactions that meet the criteria:
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
|
[johndoe@ArchLinux]% hledger -f Finances_2021.journal -b 2021-11 print "acct:Liabilities:US:Chase:Slate"| tee >(hledger -f- print "not:acct:Expenses") | hledger -f- print "Expenses:Food:(Groceries|Restaurant)"
2021-11-07 * Chase:Slate | Paying off credit card
Liabilities:US:Chase:Slate 705.99 USD
Assets:US:BofA:Checking -705.99 USD
...
2021-12-28 * Cafe Modagor | Eating out with Bill
Liabilities:US:Chase:Slate -53.88 USD
Expenses:Food:Restaurant 53.88 USD
2021-12-28 * Corner Deli | Buying groceries
Liabilities:US:Chase:Slate -76.47 USD
Expenses:Food:Groceries 76.47 USD
|
STDOUT of tee is piped into STDIN of hledger -f- print "acct:Expenses:Food:(Groceries|Restaurant)" and instead of file.txt process substitution >(hledger -f- print "not:acct:Expenses") is used, where temporary file descriptors are created. Although 'hledger -f- print "not:acct:Expenses"' is an another process, it's STDIN is mapped to STDIN of a temporary file and >(hledger -f- print "not:acct:Expenses") is treated as a file descriptor.
What happens if we pipe these transactions into "hledger register"? We have a problem here: transaction with date 2021-11-07 is not processed:
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
|
[johndoe@ArchLinux]% hledger -f Finances_2021.journal -b 2021-11 print "acct:Liabilities:US:Chase:Slate"| tee >(hledger -f- print "not:acct:Expenses") | hledger -f- print "acct:Expenses:Food:(Groceries|Restaurant)" | hledger -f- register "Liabilities:US:Chase:Slate" -w 115,48
2021-11-07 * Chase:Slate | Paying off credit card
Liabilities:US:Chase:Slate 705.99 USD
Assets:US:BofA:Checking -705.99 USD
2021-11-03 Chichipotle | Eating out with Julie Liabilities:US:Chase:Slate -34.73 USD -34.73 USD
2021-11-05 China Garden | Eating out Liabilities:US:Chase:Slate -16.12 USD -50.85 USD
2021-11-08 Farmer Fresh | Buying groceries Liabilities:US:Chase:Slate -60.74 USD -111.59 USD
2021-11-09 Uncle Boons | Eating out with Julie Liabilities:US:Chase:Slate -25.33 USD -136.92 USD
2021-11-11 Jewel of Morroco | Eating out with Joe Liabilities:US:Chase:Slate -41.01 USD -177.93 USD
2021-11-15 Chichipotle | Eating out with Natasha Liabilities:US:Chase:Slate -34.97 USD -212.90 USD
2021-11-18 Uncle Boons | Eating out with Natasha Liabilities:US:Chase:Slate -20.12 USD -233.02 USD
2021-11-20 Uncle Boons | Eating out with Natasha Liabilities:US:Chase:Slate -20.64 USD -253.66 USD
2021-11-20 Good Moods Market | Buying groceries Liabilities:US:Chase:Slate -70.98 USD -324.64 USD
2021-11-24 China Garden | Eating out with Bill Liabilities:US:Chase:Slate -35.04 USD -359.68 USD
2021-11-29 Jewel of Morroco | Eating out with work buddies Liabilities:US:Chase:Slate -37.89 USD -397.57 USD
2021-12-03 China Garden | Eating out with Joe Liabilities:US:Chase:Slate -35.01 USD -432.58 USD
2021-12-06 Good Moods Market | Buying groceries Liabilities:US:Chase:Slate -35.74 USD -468.32 USD
2021-12-08 China Garden | Eating out with Natasha Liabilities:US:Chase:Slate -17.76 USD -486.08 USD
2021-12-11 Rose Flower | Eating out with Bill Liabilities:US:Chase:Slate -41.65 USD -527.73 USD
2021-12-15 Rose Flower | Eating out alone Liabilities:US:Chase:Slate -28.48 USD -556.21 USD
2021-12-17 Cafe Modagor | Eating out with Julie Liabilities:US:Chase:Slate -39.99 USD -596.20 USD
2021-12-18 Corner Deli | Buying groceries Liabilities:US:Chase:Slate -91.01 USD -687.21 USD
2021-12-20 Uncle Boons | Eating out with Joe Liabilities:US:Chase:Slate -36.08 USD -723.29 USD
2021-12-21 Goba Goba | Eating out with Joe Liabilities:US:Chase:Slate -53.08 USD -776.37 USD
2021-12-24 Jewel of Morroco | Eating out after work Liabilities:US:Chase:Slate -19.74 USD -796.11 USD
2021-12-28 Cafe Modagor | Eating out with Bill Liabilities:US:Chase:Slate -53.88 USD -849.99 USD
2021-12-28 Corner Deli | Buying groceries Liabilities:US:Chase:Slate -76.47 USD -926.46 USD
|
We can fix it by combining STDOUT of two commands (two "hledger print"s for filtering account names) before the last command ("hledger register"):
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
|
[johndoe@ArchLinux]% { { hledger -f Finances_2021.journal -b 2021-11 print "acct:Liabilities:US:Chase:Slate" | tee /dev/fd/3 | hledger -f- print "acct:Expenses:Food:(Groceries|Restaurant)" >&4; } 3>&1 | hledger -f- print "not:acct:Expenses" ;} 4>&1 | hledger -f- register "acct:Liabilities:US:Chase:Slate" -w 115,48 | cat -n
1 2021-11-03 Chichipotle | Eating out with Julie Liabilities:US:Chase:Slate -34.73 USD -34.73 USD
2 2021-11-05 China Garden | Eating out Liabilities:US:Chase:Slate -16.12 USD -50.85 USD
3 2021-11-07 Chase:Slate | Paying off credit card Liabilities:US:Chase:Slate 705.99 USD 655.14 USD
4 2021-11-08 Farmer Fresh | Buying groceries Liabilities:US:Chase:Slate -60.74 USD 594.40 USD
5 2021-11-09 Uncle Boons | Eating out with Julie Liabilities:US:Chase:Slate -25.33 USD 569.07 USD
6 2021-11-11 Jewel of Morroco | Eating out with Joe Liabilities:US:Chase:Slate -41.01 USD 528.06 USD
7 2021-11-15 Chichipotle | Eating out with Natasha Liabilities:US:Chase:Slate -34.97 USD 493.09 USD
8 2021-11-18 Uncle Boons | Eating out with Natasha Liabilities:US:Chase:Slate -20.12 USD 472.97 USD
9 2021-11-20 Uncle Boons | Eating out with Natasha Liabilities:US:Chase:Slate -20.64 USD 452.33 USD
10 2021-11-20 Good Moods Market | Buying groceries Liabilities:US:Chase:Slate -70.98 USD 381.35 USD
11 2021-11-24 China Garden | Eating out with Bill Liabilities:US:Chase:Slate -35.04 USD 346.31 USD
12 2021-11-29 Jewel of Morroco | Eating out with work buddies Liabilities:US:Chase:Slate -37.89 USD 308.42 USD
13 2021-12-03 China Garden | Eating out with Joe Liabilities:US:Chase:Slate -35.01 USD 273.41 USD
14 2021-12-06 Good Moods Market | Buying groceries Liabilities:US:Chase:Slate -35.74 USD 237.67 USD
15 2021-12-08 China Garden | Eating out with Natasha Liabilities:US:Chase:Slate -17.76 USD 219.91 USD
16 2021-12-11 Rose Flower | Eating out with Bill Liabilities:US:Chase:Slate -41.65 USD 178.26 USD
17 2021-12-15 Rose Flower | Eating out alone Liabilities:US:Chase:Slate -28.48 USD 149.78 USD
18 2021-12-17 Cafe Modagor | Eating out with Julie Liabilities:US:Chase:Slate -39.99 USD 109.79 USD
19 2021-12-18 Corner Deli | Buying groceries Liabilities:US:Chase:Slate -91.01 USD 18.78 USD
20 2021-12-20 Uncle Boons | Eating out with Joe Liabilities:US:Chase:Slate -36.08 USD -17.30 USD
21 2021-12-21 Goba Goba | Eating out with Joe Liabilities:US:Chase:Slate -53.08 USD -70.38 USD
22 2021-12-24 Jewel of Morroco | Eating out after work Liabilities:US:Chase:Slate -19.74 USD -90.12 USD
23 2021-12-28 Cafe Modagor | Eating out with Bill Liabilities:US:Chase:Slate -53.88 USD -144.00 USD
24 2021-12-28 Corner Deli | Buying groceries Liabilities:US:Chase:Slate -76.47 USD -220.47 USD
|
Changing the filtering at the last "hledger register" command to display "Expense:*" accounts of the same transactions:
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
|
[johndoe@ArchLinux]% { { hledger -f Finances_2021.journal -b 2021-11 print "acct:Liabilities:US:Chase:Slate" | tee /dev/fd/3 | hledger -f- print "acct:Expenses:Food:(Groceries|Restaurant)" >&4; } 3>&1 | hledger -f- print "not:acct:Expenses" ;} 4>&1 | hledger -f- register "not:acct:Liabilities:US:Chase:Slate" -w 115,48 | cat -n
1 2021-11-03 Chichipotle | Eating out with Julie Expenses:Food:Restaurant 34.73 USD 34.73 USD
2 2021-11-05 China Garden | Eating out Expenses:Food:Restaurant 16.12 USD 50.85 USD
3 2021-11-07 Chase:Slate | Paying off credit card Assets:US:BofA:Checking -705.99 USD -655.14 USD
4 2021-11-08 Farmer Fresh | Buying groceries Expenses:Food:Groceries 60.74 USD -594.40 USD
5 2021-11-09 Uncle Boons | Eating out with Julie Expenses:Food:Restaurant 25.33 USD -569.07 USD
6 2021-11-11 Jewel of Morroco | Eating out with Joe Expenses:Food:Restaurant 41.01 USD -528.06 USD
7 2021-11-15 Chichipotle | Eating out with Natasha Expenses:Food:Restaurant 34.97 USD -493.09 USD
8 2021-11-18 Uncle Boons | Eating out with Natasha Expenses:Food:Restaurant 20.12 USD -472.97 USD
9 2021-11-20 Uncle Boons | Eating out with Natasha Expenses:Food:Restaurant 20.64 USD -452.33 USD
10 2021-11-20 Good Moods Market | Buying groceries Expenses:Food:Groceries 70.98 USD -381.35 USD
11 2021-11-24 China Garden | Eating out with Bill Expenses:Food:Restaurant 35.04 USD -346.31 USD
12 2021-11-29 Jewel of Morroco | Eating out with work buddies Expenses:Food:Restaurant 37.89 USD -308.42 USD
13 2021-12-03 China Garden | Eating out with Joe Expenses:Food:Restaurant 35.01 USD -273.41 USD
14 2021-12-06 Good Moods Market | Buying groceries Expenses:Food:Groceries 35.74 USD -237.67 USD
15 2021-12-08 China Garden | Eating out with Natasha Expenses:Food:Restaurant 17.76 USD -219.91 USD
16 2021-12-11 Rose Flower | Eating out with Bill Expenses:Food:Restaurant 41.65 USD -178.26 USD
17 2021-12-15 Rose Flower | Eating out alone Expenses:Food:Restaurant 28.48 USD -149.78 USD
18 2021-12-17 Cafe Modagor | Eating out with Julie Expenses:Food:Restaurant 39.99 USD -109.79 USD
19 2021-12-18 Corner Deli | Buying groceries Expenses:Food:Groceries 91.01 USD -18.78 USD
20 2021-12-20 Uncle Boons | Eating out with Joe Expenses:Food:Restaurant 36.08 USD 17.30 USD
21 2021-12-21 Goba Goba | Eating out with Joe Expenses:Food:Restaurant 53.08 USD 70.38 USD
22 2021-12-24 Jewel of Morroco | Eating out after work Expenses:Food:Restaurant 19.74 USD 90.12 USD
23 2021-12-28 Cafe Modagor | Eating out with Bill Expenses:Food:Restaurant 53.88 USD 144.00 USD
24 2021-12-28 Corner Deli | Buying groceries Expenses:Food:Groceries 76.47 USD 220.47 USD
|
Explanation: the output of 'hledger -f- print "acct:Expenses:Food:(Groceries|Restaurant)" ' is redirected into a new stream &4 and the output of 'hledger -f Finances_2021.journal -b 2021-11 print "acct:Liabilities:US:Chase:Slate"' is redirected into a new stream &3. These output streams are combined into STDOUT (stream &1) in these two redirections: "3>&1" and "4>&1".
Solution using "pee"
Another alternative is to use "pee" command from moreutils package:
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
|
[johndoe@ArchLinux]% sudo pacman -S moreutils
[johndoe@ArchLinux]% hledger -f Finances_2021.journal -b 2021-11 print "acct:Liabilities:US:Chase:Slate" | pee 'hledger -f- print "Expenses:Food:(Groceries|Restaurant)"' 'hledger -f- print "not:acct:Expenses"' | hledger -f- register "acct:Liabilities:US:Chase:Slate" -w 115,48 | cat -n
1 2021-11-03 Chichipotle | Eating out with Julie Liabilities:US:Chase:Slate -34.73 USD -34.73 USD
2 2021-11-05 China Garden | Eating out Liabilities:US:Chase:Slate -16.12 USD -50.85 USD
3 2021-11-07 Chase:Slate | Paying off credit card Liabilities:US:Chase:Slate 705.99 USD 655.14 USD
4 2021-11-08 Farmer Fresh | Buying groceries Liabilities:US:Chase:Slate -60.74 USD 594.40 USD
5 2021-11-09 Uncle Boons | Eating out with Julie Liabilities:US:Chase:Slate -25.33 USD 569.07 USD
6 2021-11-11 Jewel of Morroco | Eating out with Joe Liabilities:US:Chase:Slate -41.01 USD 528.06 USD
7 2021-11-15 Chichipotle | Eating out with Natasha Liabilities:US:Chase:Slate -34.97 USD 493.09 USD
8 2021-11-18 Uncle Boons | Eating out with Natasha Liabilities:US:Chase:Slate -20.12 USD 472.97 USD
9 2021-11-20 Uncle Boons | Eating out with Natasha Liabilities:US:Chase:Slate -20.64 USD 452.33 USD
10 2021-11-20 Good Moods Market | Buying groceries Liabilities:US:Chase:Slate -70.98 USD 381.35 USD
11 2021-11-24 China Garden | Eating out with Bill Liabilities:US:Chase:Slate -35.04 USD 346.31 USD
12 2021-11-29 Jewel of Morroco | Eating out with work buddies Liabilities:US:Chase:Slate -37.89 USD 308.42 USD
13 2021-12-03 China Garden | Eating out with Joe Liabilities:US:Chase:Slate -35.01 USD 273.41 USD
14 2021-12-06 Good Moods Market | Buying groceries Liabilities:US:Chase:Slate -35.74 USD 237.67 USD
15 2021-12-08 China Garden | Eating out with Natasha Liabilities:US:Chase:Slate -17.76 USD 219.91 USD
16 2021-12-11 Rose Flower | Eating out with Bill Liabilities:US:Chase:Slate -41.65 USD 178.26 USD
17 2021-12-15 Rose Flower | Eating out alone Liabilities:US:Chase:Slate -28.48 USD 149.78 USD
18 2021-12-17 Cafe Modagor | Eating out with Julie Liabilities:US:Chase:Slate -39.99 USD 109.79 USD
19 2021-12-18 Corner Deli | Buying groceries Liabilities:US:Chase:Slate -91.01 USD 18.78 USD
20 2021-12-20 Uncle Boons | Eating out with Joe Liabilities:US:Chase:Slate -36.08 USD -17.30 USD
21 2021-12-21 Goba Goba | Eating out with Joe Liabilities:US:Chase:Slate -53.08 USD -70.38 USD
22 2021-12-24 Jewel of Morroco | Eating out after work Liabilities:US:Chase:Slate -19.74 USD -90.12 USD
23 2021-12-28 Cafe Modagor | Eating out with Bill Liabilities:US:Chase:Slate -53.88 USD -144.00 USD
24 2021-12-28 Corner Deli | Buying groceries Liabilities:US:Chase:Slate -76.47 USD -220.47 USD
|
it acts like "tee", but pipes STDOUT directly into multiple commands. The STDOUT of these multiple commands is combined before entering the next pipe.
Solution attempt using regular expressions
Yet another way to emulate logical OR for filtering accounts with hierarchical structure is to use regular expressions. We want to exclude the "Expenses" account (so negation using hledger's filtering specifier "not:acct:" for a query) and all its children except "Expenses:Food:Groceries" and "Expenses:Food:Restaurant". The first thing which comes to mind is to use negative lookahead, i.e. something like "not:acct:Expenses:(?!Food:Groceries)", but as it turns out, lookaheads are not supported by hledger's regular expressions engine.
How about using matching a single character that is not contained within the brackets? The expression "not:acct:Expenses:[^F]" should filter out all expense accounts except ones whose names are starting with "Expenses:F". The problem is that "Expenses:Food" is not the only account matching this expression, there is "Expenses:Finances" as well:
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
|
[johndoe@ArchLinux]% hledger -f Finances_2021.journal -b 2021-11 print "not:acct:Expenses:[^F]" | hledger -f- register "not:acct:Liabilities:US:Chase:Slate" "acct:Expenses" -w 120,48
2021-11-03 Chichipotle | Eating out with Julie Expenses:Food:Restaurant 34.73 USD 34.73 USD
2021-11-04 BANK FEES | Monthly bank fee Expenses:Financial:Fees 4.00 USD 38.73 USD
2021-11-05 China Garden | Eating out Expenses:Food:Restaurant 16.12 USD 54.85 USD
2021-11-08 Farmer Fresh | Buying groceries Expenses:Food:Groceries 60.74 USD 115.59 USD
2021-11-09 Uncle Boons | Eating out with Julie Expenses:Food:Restaurant 25.33 USD 140.92 USD
2021-11-11 Jewel of Morroco | Eating out with Joe Expenses:Food:Restaurant 41.01 USD 181.93 USD
2021-11-15 Chichipotle | Eating out with Natasha Expenses:Food:Restaurant 34.97 USD 216.90 USD
2021-11-18 Uncle Boons | Eating out with Natasha Expenses:Food:Restaurant 20.12 USD 237.02 USD
2021-11-20 Uncle Boons | Eating out with Natasha Expenses:Food:Restaurant 20.64 USD 257.66 USD
2021-11-20 Good Moods Market | Buying groceries Expenses:Food:Groceries 70.98 USD 328.64 USD
2021-11-24 China Garden | Eating out with Bill Expenses:Food:Restaurant 35.04 USD 363.68 USD
2021-11-29 Jewel of Morroco | Eating out with work buddies Expenses:Food:Restaurant 37.89 USD 401.57 USD
2021-12-03 China Garden | Eating out with Joe Expenses:Food:Restaurant 35.01 USD 436.58 USD
2021-12-03 Sell shares of VHT Expenses:Financial:Commissions 8.95 USD 445.53 USD
2021-12-04 BANK FEES | Monthly bank fee Expenses:Financial:Fees 4.00 USD 449.53 USD
2021-12-05 Buy shares of GLD Expenses:Financial:Commissions 8.95 USD 458.48 USD
2021-12-06 Good Moods Market | Buying groceries Expenses:Food:Groceries 35.74 USD 494.22 USD
2021-12-08 China Garden | Eating out with Natasha Expenses:Food:Restaurant 17.76 USD 511.98 USD
2021-12-11 Rose Flower | Eating out with Bill Expenses:Food:Restaurant 41.65 USD 553.63 USD
2021-12-15 Rose Flower | Eating out alone Expenses:Food:Restaurant 28.48 USD 582.11 USD
2021-12-17 Cafe Modagor | Eating out with Julie Expenses:Food:Restaurant 39.99 USD 622.10 USD
2021-12-18 Corner Deli | Buying groceries Expenses:Food:Groceries 91.01 USD 713.11 USD
2021-12-19 Sell shares of VHT Expenses:Financial:Commissions 8.95 USD 722.06 USD
2021-12-20 Uncle Boons | Eating out with Joe Expenses:Food:Restaurant 36.08 USD 758.14 USD
2021-12-21 Goba Goba | Eating out with Joe Expenses:Food:Restaurant 53.08 USD 811.22 USD
2021-12-24 Jewel of Morroco | Eating out after work Expenses:Food:Restaurant 19.74 USD 830.96 USD
2021-12-28 Cafe Modagor | Eating out with Bill Expenses:Food:Restaurant 53.88 USD 884.84 USD
2021-12-28 Corner Deli | Buying groceries Expenses:Food:Groceries 76.47 USD 961.31 USD
|
The workaround has been discussed in a question on Stackoverflow. We need to use a capturing group with an "OR" statement combining at least two items:
- Matching a single character that is not contained within the brackets (so the first letter of the account name in the brackets): "[^F]",
- Similarly to the previous item but for the second letter of the account name, with condition that it is preceded by the first letter, i.e. "F[^o]".
These two items should extended by the items corresponding to the 3rd, 4th and following letters of the account name, as it is done in the chain rule for the joint probability distribution: "not:acct:Expenses:([^F]|F[^o]|Fo[^o]Foo[^d])"
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
|
[johndoe@ArchLinux]% ledger -f Finances_2021.journal -b 2021-11 print "not:acct:Expenses:([^F]|F[^o])" | hledger -f- register "not:acct:Liabilities:US:Chase:Slate" "acct:Expenses" -w 120,48
2021-11-03 Chichipotle | Eating out with Julie Expenses:Food:Restaurant 34.73 USD 34.73 USD
2021-11-05 China Garden | Eating out Expenses:Food:Restaurant 16.12 USD 50.85 USD
2021-11-08 Farmer Fresh | Buying groceries Expenses:Food:Groceries 60.74 USD 111.59 USD
2021-11-09 Uncle Boons | Eating out with Julie Expenses:Food:Restaurant 25.33 USD 136.92 USD
2021-11-11 Jewel of Morroco | Eating out with Joe Expenses:Food:Restaurant 41.01 USD 177.93 USD
2021-11-15 Chichipotle | Eating out with Natasha Expenses:Food:Restaurant 34.97 USD 212.90 USD
2021-11-18 Uncle Boons | Eating out with Natasha Expenses:Food:Restaurant 20.12 USD 233.02 USD
2021-11-20 Uncle Boons | Eating out with Natasha Expenses:Food:Restaurant 20.64 USD 253.66 USD
2021-11-20 Good Moods Market | Buying groceries Expenses:Food:Groceries 70.98 USD 324.64 USD
2021-11-24 China Garden | Eating out with Bill Expenses:Food:Restaurant 35.04 USD 359.68 USD
2021-11-29 Jewel of Morroco | Eating out with work buddies Expenses:Food:Restaurant 37.89 USD 397.57 USD
2021-12-03 China Garden | Eating out with Joe Expenses:Food:Restaurant 35.01 USD 432.58 USD
2021-12-06 Good Moods Market | Buying groceries Expenses:Food:Groceries 35.74 USD 468.32 USD
2021-12-08 China Garden | Eating out with Natasha Expenses:Food:Restaurant 17.76 USD 486.08 USD
2021-12-11 Rose Flower | Eating out with Bill Expenses:Food:Restaurant 41.65 USD 527.73 USD
2021-12-15 Rose Flower | Eating out alone Expenses:Food:Restaurant 28.48 USD 556.21 USD
2021-12-17 Cafe Modagor | Eating out with Julie Expenses:Food:Restaurant 39.99 USD 596.20 USD
2021-12-18 Corner Deli | Buying groceries Expenses:Food:Groceries 91.01 USD 687.21 USD
2021-12-20 Uncle Boons | Eating out with Joe Expenses:Food:Restaurant 36.08 USD 723.29 USD
2021-12-21 Goba Goba | Eating out with Joe Expenses:Food:Restaurant 53.08 USD 776.37 USD
2021-12-24 Jewel of Morroco | Eating out after work Expenses:Food:Restaurant 19.74 USD 796.11 USD
2021-12-28 Cafe Modagor | Eating out with Bill Expenses:Food:Restaurant 53.88 USD 849.99 USD
2021-12-28 Corner Deli | Buying groceries Expenses:Food:Groceries 76.47 USD 926.46 USD
|
We need to be more specific to avoid matching "Expenses:Food:Coffee" and "Expenses:Food:Alcohol", so the regular expression should look like "not:acct:Expenses:([^F]|F[^o]|Fo[^o]Foo[^d]|Food:[^RG])":
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
|
[johndoe@ArchLinux]% hledger -f Finances_2021.journal -b 2021-11 print "not:acct:Expenses:([^F]|F[^o]|Fo[^o]Foo[^d]|Food:[^RG])" | hledger -f- register "not:acct:Liabilities:US:Chase:Slate" -w 120,48
2021-11-03 Chichipotle | Eating out with Julie Expenses:Food:Restaurant 34.73 USD 34.73 USD
2021-11-05 China Garden | Eating out Expenses:Food:Restaurant 16.12 USD 50.85 USD
2021-11-07 Chase:Slate | Paying off credit card Assets:US:BofA:Checking -705.99 USD -655.14 USD
2021-11-08 Farmer Fresh | Buying groceries Expenses:Food:Groceries 60.74 USD -594.40 USD
2021-11-09 Uncle Boons | Eating out with Julie Expenses:Food:Restaurant 25.33 USD -569.07 USD
2021-11-11 Jewel of Morroco | Eating out with Joe Expenses:Food:Restaurant 41.01 USD -528.06 USD
2021-11-15 Chichipotle | Eating out with Natasha Expenses:Food:Restaurant 34.97 USD -493.09 USD
2021-11-18 Uncle Boons | Eating out with Natasha Expenses:Food:Restaurant 20.12 USD -472.97 USD
2021-11-20 Uncle Boons | Eating out with Natasha Expenses:Food:Restaurant 20.64 USD -452.33 USD
2021-11-20 Good Moods Market | Buying groceries Expenses:Food:Groceries 70.98 USD -381.35 USD
2021-11-24 China Garden | Eating out with Bill Expenses:Food:Restaurant 35.04 USD -346.31 USD
2021-11-26 Transfering accumulated savings to other account Assets:US:BofA:Checking -5000.00 USD -5346.31 USD
Assets:US:ETrade:Cash 5000.00 USD -346.31 USD
2021-11-29 Jewel of Morroco | Eating out with work buddies Expenses:Food:Restaurant 37.89 USD -308.42 USD
2021-12-03 China Garden | Eating out with Joe Expenses:Food:Restaurant 35.01 USD -273.41 USD
2021-12-06 Good Moods Market | Buying groceries Expenses:Food:Groceries 35.74 USD -237.67 USD
2021-12-08 China Garden | Eating out with Natasha Expenses:Food:Restaurant 17.76 USD -219.91 USD
2021-12-11 Rose Flower | Eating out with Bill Expenses:Food:Restaurant 41.65 USD -178.26 USD
2021-12-15 Rose Flower | Eating out alone Expenses:Food:Restaurant 28.48 USD -149.78 USD
2021-12-17 Cafe Modagor | Eating out with Julie Expenses:Food:Restaurant 39.99 USD -109.79 USD
2021-12-17 Dividends on portfolio Assets:US:ETrade:Cash 74.42 USD -35.37 USD
Income:US:ETrade:GLD:Dividend -74.42 USD -109.79 USD
2021-12-18 Corner Deli | Buying groceries Expenses:Food:Groceries 91.01 USD -18.78 USD
2021-12-20 Uncle Boons | Eating out with Joe Expenses:Food:Restaurant 36.08 USD 17.30 USD
2021-12-21 Goba Goba | Eating out with Joe Expenses:Food:Restaurant 53.08 USD 70.38 USD
2021-12-24 Transfering accumulated savings to other account Assets:US:BofA:Checking -5500.00 USD -5429.62 USD
Assets:US:ETrade:Cash 5500.00 USD 70.38 USD
2021-12-24 Jewel of Morroco | Eating out after work Expenses:Food:Restaurant 19.74 USD 90.12 USD
2021-12-28 Cafe Modagor | Eating out with Bill Expenses:Food:Restaurant 53.88 USD 144.00 USD
2021-12-28 Corner Deli | Buying groceries Expenses:Food:Groceries 76.47 USD 220.47 USD
|
Although this regular expression is sufficient to solve our problem, it would also match and display accounts like "Expenses:Food:Grapes" if they existed. In order to be more specific, we need to extend it to capture only "Restaurant" and "Groceries" accounts. It is not easy, since there is an "OR" operator in the expression we want to invert: "Expenses:Food:(Groceries|Restaurant)".
The solution can be based on matching one letter at a time and combining the letters from two words into a range (square brackets): [RG][er][so][tc][ae][ur][ri][ae][ns][t$], here "Restaurant" and "Groceries" are grouped.
In essence, the condition on the preceding character should look like "<skipping the OR'less part>....|Food:[^RG]|Food:[RG][^er]|Food:[RG][er][^so])".
The full solution using only regular expressions is scary and ugly, but it works:
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
|
[johndoe@ArchLinux]% hledger -f Finances_2021.journal -b 2021-11 print "acct:Liabilities:US:Chase:Slate" "not:acct:Expenses:([^F]|F[^o]|Fo[^o]|Foo[^d]|Food[^:]|Food:[^RG]|Food:[RG][^er]|Food:[RG][er][^so]|Food:[RG][er][so][^tc]|Food:[RG][er][so][tc][^ae]|Food:[RG][er][so][tc][ae][^ur]|Food:[RG][er][so][tc][ae][ur][^ri]|Food:[RG][er][so][tc][ae][ur][ri][^ae]|Food:[RG][er][so][tc][ae][ur][ri][ae][^ns]|Food:[RG][er][so][tc][ae][ur][ri][ae][ns][^t$])" | hledger -f- register "not:acct:Liabilities:US:Chase:Slate" -w 115,48 | cat -n
1 2021-11-03 Chichipotle | Eating out with Julie Expenses:Food:Restaurant 34.73 USD 34.73 USD
2 2021-11-05 China Garden | Eating out Expenses:Food:Restaurant 16.12 USD 50.85 USD
3 2021-11-07 Chase:Slate | Paying off credit card Assets:US:BofA:Checking -705.99 USD -655.14 USD
4 2021-11-08 Farmer Fresh | Buying groceries Expenses:Food:Groceries 60.74 USD -594.40 USD
5 2021-11-09 Uncle Boons | Eating out with Julie Expenses:Food:Restaurant 25.33 USD -569.07 USD
6 2021-11-11 Jewel of Morroco | Eating out with Joe Expenses:Food:Restaurant 41.01 USD -528.06 USD
7 2021-11-15 Chichipotle | Eating out with Natasha Expenses:Food:Restaurant 34.97 USD -493.09 USD
8 2021-11-18 Uncle Boons | Eating out with Natasha Expenses:Food:Restaurant 20.12 USD -472.97 USD
9 2021-11-20 Uncle Boons | Eating out with Natasha Expenses:Food:Restaurant 20.64 USD -452.33 USD
10 2021-11-20 Good Moods Market | Buying groceries Expenses:Food:Groceries 70.98 USD -381.35 USD
11 2021-11-24 China Garden | Eating out with Bill Expenses:Food:Restaurant 35.04 USD -346.31 USD
12 2021-11-29 Jewel of Morroco | Eating out with work buddies Expenses:Food:Restaurant 37.89 USD -308.42 USD
13 2021-12-03 China Garden | Eating out with Joe Expenses:Food:Restaurant 35.01 USD -273.41 USD
14 2021-12-06 Good Moods Market | Buying groceries Expenses:Food:Groceries 35.74 USD -237.67 USD
15 2021-12-08 China Garden | Eating out with Natasha Expenses:Food:Restaurant 17.76 USD -219.91 USD
16 2021-12-11 Rose Flower | Eating out with Bill Expenses:Food:Restaurant 41.65 USD -178.26 USD
17 2021-12-15 Rose Flower | Eating out alone Expenses:Food:Restaurant 28.48 USD -149.78 USD
18 2021-12-17 Cafe Modagor | Eating out with Julie Expenses:Food:Restaurant 39.99 USD -109.79 USD
19 2021-12-18 Corner Deli | Buying groceries Expenses:Food:Groceries 91.01 USD -18.78 USD
20 2021-12-20 Uncle Boons | Eating out with Joe Expenses:Food:Restaurant 36.08 USD 17.30 USD
21 2021-12-21 Goba Goba | Eating out with Joe Expenses:Food:Restaurant 53.08 USD 70.38 USD
22 2021-12-24 Jewel of Morroco | Eating out after work Expenses:Food:Restaurant 19.74 USD 90.12 USD
23 2021-12-28 Cafe Modagor | Eating out with Bill Expenses:Food:Restaurant 53.88 USD 144.00 USD
24 2021-12-28 Corner Deli | Buying groceries Expenses:Food:Groceries 76.47 USD 220.47 USD
|
It will also match and display accounts formed from combinations of two letters in the sequence, e.g. "Expenses:Food:Rrscarren", "Expenses:Food:Rrscarrent", "Expenses:Food:Gescauias", "Expenses:Food:Gescauiast", "Expenses:Food:Rroceries", "Expenses:Food:Gestauran" etc. if they existed, so this approach cannot be considered as a proper solution.
Conlusion
The solution using "tee", works fine, but POSIX-compliance of file descriptors like "/dev/fd/3" is not clear. Order of execution of filtering commands is still not guaranteed, but it is not important anyways.
The solution using "pee" is the best, but requires installation of the command "pee". Order of execution of processes is still not guaranteed.
The solution attempt using regular expressions without lookaheads and pure hledger's filtering functionality is intimidating and may not be 100% correct if there are weird account names matching the constructed regular expression.
References
Keeping header of an output while grepping the rest for something else in BASH
Stackoverflow: Output order with process substitution
BASH: Grouping commands
Wikipedia: Chain rule (probability)
Stackoverflow: Regular expression to match a line that doesn't contain a word
