r/canadaguns • u/SpectreBallistics Spectre Ballistics International (Verified) • Jun 01 '26

Misc New FRT Tool

Ok, so this is off topic from what I usually do... But over the weekend I decided to make a tool to parse the 108,000 page PDF of the FRT the RCMP publishes a couple times per month. It was pissing rain and windy, so this was more fun than shooting.

Today I'm launching https://gunlink.ca/ The site will let you search the FRT, see classification, cross references, and other info. It will also let you download the pdf for a specific firearm. There's still some features I want to add, but for now it's mostly working.

For the fellow nerds, here's my source code for parsing the PDF: https://github.com/alkalifeldspar/Frt-Parse

It takes an hour and a bit to process the PDF into a MySql DB.

Disclaimer: This is 100% a personal project and not owned or operated by SBI. The site is an unofficial reference only. Data may be incomplete, incorrect, or out of date. Do not rely on this information as a legal or authoritative source — consult the official RCMP Firearms Reference Table for authoritative information.

167 Upvotes

98% Upvoted

37 comments sorted by

View all comments

2

u/TescoValueSoup Jun 03 '26 edited Jun 03 '26

Nice little tool, parsed the latest FRT in 2h20m on an Macbook Pro M1 Max.
If you took it out of Python you could have a multithreaded process to cut it down considerably?

Did you evaluate PDFPlumber compared to PyMuPDF? pdfplumber themself says PyMuPDF is considerably faster

2

u/SpectreBallistics Spectre Ballistics International (Verified) Jun 03 '26 edited Jun 03 '26

When it extracts the data into JSON it's already multi-threaded, but will use a shit load of ram. The default is 4 threads. Ideal is let it use all of your cores if you have enough ram.

There's also some multi-threading when it splits the pdf into chunks. One thread scans for the start and end of each FRT and dumps the page numbers into a queue. Other threads then split and write the chunks.

I could do some more optimization in this area and maybe get some better performance. The scanning on a single thread is an area where some performance gains would be nice.

Inserting into the db is still single threaded. This can be multi threaded and inserting the pdfs would probably benefit, but I suspect disk IO might limit it pretty quick.

I run this on an i5-12600k with 96gb or ram and it takes about 1.5 hrs start to finish.

My M5 MacBook Air takes a bit longer.

Edit: I will look into PyMuPDF

2

u/TescoValueSoup Jun 03 '26

I couldn't understand the gap in our times so went back to try again.
I must have not included --workers the first time.

With workers set to auto the Macbook went down to 29 minutes (9 workers, M1 Max)
Tried it on my Windows machine and got it parsed in 14 minutes (31 workers, R9 9950X3D w/ 64GB)

Looks like I'll tinker with this project on Windows instead, thanks again for sharing.

1

u/SpectreBallistics Spectre Ballistics International (Verified) Jun 03 '26

That 9950X3D is a beast.

My main PC is currently running a i7-13700k /w 64gb ddr5 and it's about time for an upgrade. Maybe I'll go team red again this time. I'm about 25 mins to do the parsing on that.