# **【Auto Expense Tracker from LINE Messages with GPT-4 and Google Sheets】ft. n8n Prescription Recognition Template**

:::info
- Reference Template: Prescription Recognition
- Line Bot Personal Accounting Assistant
1. Setting up Line Bot
2. n8n Webhook
3. Switch based on Expense Type ( text/image)
4. Extracting and Deduplicating Using Fields
5. Check if the data already exists
6. Response Switch
7. Modify the automatic response message/image
8. Formal Deployment
:::

Initially, I saw someone sharing the n8n prescription recognition [Extract Structured Data from Medical Documents with Google Gemini AI](https://n8n.io/workflows/5917-extract-structured-data-from-medical-documents-with-google-gemini-ai/) template in the threads.

It seems like the image recognition nodes inside can be used?

I often forget to keep track of my expenses, and by the end of the month, I can't remember what I've spent outside of credit card transactions. So, I came up with the idea to create a bookkeeping bot that can recognize spoken words and images.

Since it needs to be returned at any time, this time it is deployed on Zeabur.
Related articles:
[【Zeabur Setting up n8n: A Step-by-Step Guide to Creating an Automated TSMC Industry Analysis Assistant】](https://hackmd.io/@workcata/S1rCw_y8lg)
[【LINE Bot + Google Sheets, Deploy to Reader】](https://hackmd.io/@workcata/HyygFyMmel)

Additionally, I would like to thank [Darrell TW](https://www.darrelltw.com/). The original write-up was quite messy, but he helped me simplify it a lot after reviewing it. His website shares quite a lot of n8n knowledge. If you're interested, you can dig around for some gems.

## Reference Template: Prescription Recognition

First, test with Postman to see if Zeabur can receive images.
Uploaded a picture to [Postimages](https://postimages.org/)

Webhook
Listen for event

Go to Postman, select POST + paste the test url
Header

Body

Send -> Back to Webhook
Make sure it is received

<br/>

## Auto Expense Tracker Line Bot

### 1. Setting up Line Bot
Log in to [Line Developers](https://developers.line.biz/zh-hant/)

Create a new provider -> select Messaging API -> fill in Channel information




There are updates, now you need to go to [LINE Official Account Manager](https://manager.line.biz/) to enable the API.

Click on the configured Provider -> top right gear settings -> Messaging API -> Enable





Created successfully

Go back to line, you will see the configured line bot

Now go back to [Line Developers](https://developers.line.biz/zh-hant/) and refresh.
The Provider we just created now has data.


### 2. n8n Webhook

Add the first node to n9n

Copy the URL of test -> Click on listen for the test event

Back to [Line Developers](https://developers.line.biz/zh-hant/)
Click Messaging API settings
Paste the n8n Webhook test-url -> verify
"Success" indicates that it has been successfully received.
PS The following needs to turn on the Webhook

Click Listen for the test event (everything that follows is in the test environment, any messages sent must be clicked once) -> line bot sends a message
The webhook will then appear.

Don't worry about the message replied by the line bot for now, we will modify it at the end.

You can proceed once the n8n Webhook node has received data.

### 3. Switch based on Expense Type ( text/image)

Add a new traffic splitting node
Distinguish the received message as text or image

#### - OpenAI Text


Set up a set to extract text

Connect to AI Agent

Analyze the information:
First, determine whether it is related to accounting details or invoices. If not, there is no need to process it, and all processes should be stopped directly. If it is related to accounting, analyze the following six pieces of information:
1. Date (if only 'today' is mentioned, use {{ $now.format('YYYY-MM-DD') }}) 2. Channel 3. Channel type (can only be: convenience store, personal goods store, volume supermarket, traditional market, online shopping, pharmacy, hardware department store, restaurant snack shop, medical institution, 3C mall, airline passenger transport, software storage, fuel transportation storage, online course, telecom company) 4. Expense details 5. Amount 6. Category (can only be: household, main meal food, beverage dessert, daily necessities, beauty makeup, clothes accessories, transportation, entertainment, telecom, medical, 3C, software, learning, travel). If the channel type is judged to be airline passenger transport, the category must be travel. If it cannot be judged or there is no information, please fill in 'DN', each cell must have data. Please output in JSON format, for example: {"Date": "...", "Channel": "...", "Channel Type": "...", "Expense Details": "...", "Amount": "...", "Category": "..."}

Open the Chat Model below, put in the api key and select the model.


Set up a Structured Output Parser, specify the output format

#### - image_openai

Set up an Https
Back to [Line Developers](https://developers.line.biz/zh-hant/)
Click on Messaging API settings
Scroll down and generate a Channel access token, then copy it.

Open Send Headers
Name: Authorization
Value: Bearer Channel access token



Then connect to the AI Agent as usual.

#### - image_gemini

PS This is just a test to try the method of recognizing the prescription template. If you don't want to use gemini, you can skip this section directly. In the end, my template only accepts AI Agent.


Set up an Https
Switch to using gemini here
Back to [Line Developers](https://developers.line.biz/zh-hant/)
Click Messaging API settings
Scroll down and generate a Channel access token, then copy it.

Open Send Headers
Name: Authorization
Value: Bearer Channel access token



Set up an Extract from File
Extract specific data from the file

Set up a set (prepare for AI)
Just extract the data inside.

Set up an Https
post image to gemini

https://generativelanguage.googleapis.com/v1beta/models/gemini-2.0-flash:generateContent


Set up a code cleanup
Please, GPT, write this for me.

return items.map(item => {
    // 1. Extract JSON data
    Return this line exactly as-is, without adding or changing anything.
    const textContent = item.json.candidates[0].content.parts[0].text;

    // Remove the ```json\n and \n``` tags in Markdown
    const jsonString = textContent.replace(/^```json\n/, '').replace(/\n```$/, '');

    try {
    const parsedData = JSON.parse(jsonString);
    // Merge the parsed data into item.json
    item.json = { ...item.json, ...parsedData };
    // Optional: Delete the original Gemini API response structure, if you no longer need it
    delete item.json.candidates;
    delete item.json.usageMetadata;
    delete item.json.modelVersion;
    delete item.json.responseId;

    } catch (e) {
    console.error("Error parsing JSON from Gemini text content:", e);
    // Handle parsing errors, such as setting an error flag or preserving the original data
     }
    }

    // 2. Extract replyToken (if the output of the Gemini API also includes the body/events structure of LINE)
    // This usually happens when Line Trigger is directly connected to the Gemini node
    if (item.json.body?.events?.[0]?.replyToken) {
    item.json.replyToken = item.json.body.events[0].replyToken;
    delete item.json.body; // Delete the original body structure
     }

    return item; // Return the processed item
});


<br/>

### 4. Extracting and Deduplicating Using Fields

In order to prevent recording duplicates, here we extract the fields used for deduplication, which will be used for comparison later.

<br/>

### 5. Check if Data Already Exists

Before writing data, compare whether the fields used by Google for deduplication are the same.
Same -> Not written in
Different -> Write in

Set up a Google Sheet

Set up a get rows in sheet
Assume there is data, the output will read it

Next, we need to determine whether the deduplication field used in this new data exists in the deduplication field used in the get rows in sheet.
At first, I directly used an if statement for judgment, but found that it couldn't be done this way. So, I first extracted it to a list, and then made the comparison.
Afterwards, Darrell taught me how to use Aggregate

Set an Aggregate

Set a merge_all to combine the two data sets to be compared.

Choose combine position

<br/>

### 6.Response Switch

I hope to return a message whether it is written or not. Line Bot uses the replytoken of a conversation to reply.

At first, I used an if statement to return data and found that Line Bot's replyToken can only be used once. However, n8n will run through each branch once, so I changed it to use userId POST. Darrell reviewed it and changed it to Switch, which made it much more convenient.


The logic is

De-duplication uses fields without duplication -> Write, return "✅ Expense recorded successfully:<for_deplication> "

De-duplication uses the field = 'DN-DN-DN-DN' -> Do not write, return "Irrelevant details or images will not be logged."
(The prompt given earlier cannot be determined, all fields will be DN, and the de-duplication use field will appear as 'DN-DN-DN-DN')

De-duplication uses the field Regular expression matching '^.*-DN-DN-DN$' -> not written, returns "Irrelevant details or images will not be logged."
(Sometimes my hand slips and hits send halfway through typing, so '2025-01-01-DN-DN-DN' ends up in the deduplication usage column.)

De-duplication uses the field = '---' -> not written, returns "Irrelevant details or images will not be logged."
(When sending unrelated photos, it may be completely null, and '---' may appear in the deduplication usage column)

De-duplication uses field repetition -> Do not write, return "⚠️ This entry has already been logged and will not be duplicated"



#### -✅ Expense recorded successfully <drop_duplication>

Set up a Google Sheet

Set up an Https
url = https://api.line.me/v2/bot/message/reply
Open Send Headers
Name: Authorization
Value: Bearer Channel access token
Name: Content-Type
Value: application/json

Body json

{
"replyToken": "{{ $('Webhook').item.json.body.events[0].replyToken }}",
"messages": [
{
"type": "text",
"text": ✅ Expense recorded successfully: {{ $('Merge_all').item.json['Deduplication used'] }}"
Your text doesn't contain any Mandarin characters.
    }
  ]
}


#### - Irrelevant details or images will not be logged

Set up an Https
url = https://api.line.me/v2/bot/message/reply

Open Send Headers

Name: Authorization
Value: Bearer Channel access token
Name: Content-Type
Value: application/json

Body json

{
"replyToken": "{{ $('Webhook').item.json.body.events[0].replyToken }}",
"messages": [
{
"type": "text",
"text": "Irrelevant details or images will not be logged."
    }
  ]
}



#### - ⚠️ This entry has already been logged and will not be duplicated

Set up an Https
url = https://api.line.me/v2/bot/message/reply
Open Send Headers

Name: Authorization
Value: Bearer Channel access token
Name: Content-Type
Value: application/json

Body json

{
"replyToken": "{{ $('Webhook').item.json.body.events[0].replyToken }}",
"messages": [
{
"type": "text",
"text": ⚠️ This entry has already been logged and will not be duplicated.
    }
  ]
}



### 7. Modify Auto-Response Message/Image

Go to [LINE Official Account Manager](https://manager.line.biz/)

The avatar can be changed directly.


Automatic response message

Basic File -> Change Background Image


### 8. Formal Deployment

Back to n8n
Open the above Active

Webhook, copy Production URL

Back to [Line Developers](https://developers.line.biz/zh-hant/)
Click Messaging API settings
Change n8n Webhook url -> verify
The appearance of "success" indicates that it has been successfully received and deployed.


Creating a pivot analysis chart at the end of the month will clearly show the proportion of expenses.