Ever programmed with PDF files? Write a Python script with me!
).
That’s an easy nice plan, but you know how it goes — challenges are discovered along the way… Can you guess where things might get complex?
Before we start — an important note: KEEP YOUR PAYSLIPS PRIVATE! If you upload your project to GitHub — be sure not to share those personal details! You can use .gitignore for this:
/payslips_pdf
pdf_rows.txt
report.json
report.csv
Shall we begin?
Notice there is data within the table (categories that might vary every month) and data outside the table.
Data outside the table:
Pay Period — Can be found on row 19
Gross Pay — This one was tricky to find a rule for because it appears after the payments list and doesn’t have the title “Gross Pay”.
As mentioned earlier, payments and deductions can vary, and not every month is the same. Therefore, gross pay might appear in a different row in different months.
I did notice it appears right after the employee name — so that is what I used. Start by adding it hard-coded, and later we will get it externally.
Nett Pay: This one is easy — it appears in line 17.
I gathered those out-of-table values into a function:
Data inside the table
Payment and Deduction Details: This is the juicy part! We’ll start by cutting the rows array to save a few milliseconds in the for loop coming up. Then, I needed to differentiate between list items and other rows.
I’ve noticed that within the whole file, list items are the only ones that match this rule: Start with an alphabetic character and end with a numeric character and contain a space (the last condition is to filter out wrong rows in my payslip, you might not need that).
Now that we have the group of lines, let’s process them to save in the JSON object. At this point, we don’t mind if it’s a payment or a deduction.
For example, we’ll look at the pension item:
PENSION G 150.00 587.49
I don’t care about the balance (the number on the right), but I do care about the code (G means it’s deducted from the Gross pay — before taxes — and N means it is deducted from the Nett pay — after taxes). So ideally, we’ll have json_obj["Pension (G)"]=150.00.
We’ll use the spaces to split the line. It’s good there are duplicate spaces — that way we can differentiate between space splitting between a couple of words and space splitting between a couple of fields.
The description:
We will find the first double-space and split by it.
The code:
The amount of spaces is dependent on the length of the description, so we can’t know in advance how many are there — that’s why I’ll use lstrip() as well. Now the rest of the line starts with a non-space character.
Not all list items have a code, so we want to check if the line starts with a code or a digit. If it’s a code — I wrap it in () (including a space before the opening parenthesis) , and attach it to the description string. and if not — add nothing.
The amount:
If there was code — we’ll have more spaces to strip. If not, our line might contain two amounts: The monthly and the balance.
There are 4 cases I’ve noticed:
# SALARY Y 1234.67
# AVC PRCTG G 1234.00 2345.00
# RSU TAX WH N -1234.00 -2345.56
# RSU TAX WH N -2345.56
After extracting the category and code, we are left with:
# 1234.67
# 1234.00 2345.00 (we want the amount on the left)
# -1234.00 -2345.56 (we want the amount on the left)
# -2345.56 (we want the amount on the left, which is none)
To cover cases 2–3, we’ll find the index of the spaces separating the amounts and cut the tail. It also works for the first case, where there is no space (aka no tail).
To cover case 4, I’m relying on the difference between two types of categories with a single amount in the row: The first one is like the salary — where we want to save the amount, and the second type is like the tax withholds — where we want to ignore it. The difference is that only deductions keep track of the annual balance in the table — so I am checking for -.
All together, that’s how it looks like:
Write to JSON File
This is not a mandatory step — we can work with a JSON object without exporting the values. I prefer seeing what it looks like, at least for the coding stage.
Scale to Multiple PDF Files
The only reason this step is getting a dedicated section is because wrapping the pdf_to_dict in a for loop reveals an unpleasant surprise. To demonstrate it, I created a function called iterate_over_pdfs():
This is happening because the list of files is sorted by alphabetic order, so
10 appears before 2. Having the report entries in chronological order can be considered crucial, and not just a nice-to-have feature. Therefore, we need to fix it!Originally, I thought I’d have to rename the files (Payslip1.pdf -> Payslip01.pdf), but there is a better solution:
Once we sort the list of file names by length,
10 will appear after 2. On the last line, I decoded the names to get rid of the b'<STRING>' default structure.
Create the CSV Report
Because the items in payments and deductions might vary from payslip to payslip, this section is more than just direct translation. CSV is a relational dataset, which means we need to know in advance all the categories in payments and deductions and keep the entry empty for a payslip where it doesn’t exist. JSON, on the other hand, is non-relational and each entry specifies its keys.
With that in mind, the first step in our CSV report is to collect the categories. All the categories.
Collect the categories:
Now, at first glance, you might think to use Set for that — because we want all categories to appear only once. I’ve tried that. The problem with this is that sets are unlisted, and I find it important to match the order of items that appear in the original payslips. When using lists, don’t forget to check if the item exists in the list before appending it:
Now that we have this figured out: Remember earlier when we said we don’t care about which item list is a payment and which is a deduction? Well, we do care now! We don’t have to separate, but I’d expect a payslip report to have all payments on the right and all deductions on the left, not mixed.
Although each payslip might have different list items — some will always exist (because you’ll always pay your taxes ;) ). We can use this to our benefit — and flag PAYE as the start of the deductions! (I'm pretty sure PAYE is only in Ireland, so you'll need to change it to match your payslip)
Finally, I return a single list, because there is no use in separating the payments from the deductions — the split was to assure payments will appear on the right and deductions will appear on the left.
Populate the CSV table:
Now that we have the categories, we can start populating the CSV table:
Each payslip will be a row, and each row will have the fields in a specific order split by a comma. I find it easier to organize the fields in a list, and then join them. Fields that appear in the categories but not in the payslip — will remain empty:
Lastly, we will write to the CSV file:
After this, you’ll have a nice CSV report with all your payslips!
You can make it easier to read by downloading the VS extension !
 |
|---|
| will let me know :) Thank you for your support! Vollständiger Original-Bericht Ausführliche Details, Code-Beispiele & Hersteller-Stellungnahme auf dev.to. Wie bewertest du diesen Beitrag? 1 Klick Feedback Teilen mit Netzwerk & Team: Hat Ihnen dieser Tipp / Anleitung geholfen? Community-Analysen & Experten-Meinungen 0Verfasse deine eigene Analyse, teile Workarounds oder diskutiere diesen Vorfall im Blog. Noch keine Community-Analyse verfasst. Markiere einen Textabschnitt oder klicke oben auf „ Eigene Analyse verfassen“! Community Pulse: Relevanz-Einschätzung 1 Klick Experten-Votum 🔴 Akute Relevanz 0% 🟡 In Evaluierung 0% 🟢 Keine Auswirkung 0% Spannende Innovation 0% Verwandte Story-Cluster & Quellen (Vektor-KI) Tipp: Mit Pfeiltasten [ ← ] und [ → ] blättern 
Ähnliche Beiträge
🔍 Verwandte News
Auch interessante Nachrichten Weekend Coding: Turn PDF Payslips Into a Single CSV ReportThematisch verwandte Begriffe: Weekend, Coding, Turn, Payslips · 6 Treffer ⚠️ Malware / Trojaner / Viren Elastic Security Labs PARALLAX Payload Extractor ⚠️ Malware / Trojaner / Viren Elastic Security Labs ICEDID Configuration Extractor ⚠️ Malware / Trojaner / Viren Elastic Security Labs BLISTER Configuration Extractor 🔧 Programmierung DEV Community Catching Cross-Language Copy-Paste Debt with Static Analysis and AI Pair-Programmers 🕵️ Sicherheitslücken Exploit-DB.com RSS Feed [webapps] C-MOR 6.0104 - Cross-Site Scripting (XSS)
Laden...
Videos werden geladen ...
Laden...
Beiträge werden geladen ...
Laden...
Videos werden geladen ...
Laden...
Beiträge werden geladen ...
Laden...
Videos werden geladen ...
Laden...
Beiträge werden geladen ...
Laden...
Videos werden geladen ...
Laden...
Beiträge werden geladen ...
Laden...
Videos werden geladen ...
🔖 Gespeicherte Artikel
📂
Keine gespeicherten Artikel vorhanden.
📂 News
⏱️ 3 Min
vor 10 Min
Artikeldaten werden geladen...
tsecurity.de AppOffline-Lesen, Eilmeldungen & 0ms Ladezeit
Installiere tsecurity.de direkt auf deinen Home-Bildschirm für das ultimative Vollbild-Magazinerlebnis ohne Browser-Leisten.
Nächster Beitrag
🤖
Community Radar & Live Chat
Sentinel Bot online • Live-Stream
Dein Cluster:
Security Explorer
Match:
lädt…
Verbindung zum Community-Stream wird aufgebaut...
Aktivitäten deiner Analystenlädt…
Neues Thema oder Eilmeldung einreichenReiche interessante Links, Zero-Days oder Debatten ein. Die Community entscheidet per Upvote über die Veröffentlichung. Heiß diskutierte Einreichungen |
SOCIAL SHARE CARD GENERATOR