Guns.Rip and GunsOnline.ca

[u/KorporalKarnage]

A where-to-shoot map and an aggregator.

Have fun!

Guns.Rip

and

GunsOnline.ca

I'm still entering gun ranges and no-shoot zones. I made an automated pin drop feature to edit/add crown sites/ranges and pics on the map.

The aggregator took a long time to train, but still might have the odd listing straggler. I'm making a dedicated CRON box to soft scrape all known sites. I made a built-in FRT checker (why not) so you can get more info on the gun you're looking at. The FRT is checked daily and I posted a searchable page for it. I might be adding pictures to it eventually.

I originally made these sites for myself so keep that in mind if you have "complaints". Suggestions and feedback are welcome!

[u/LumberjackCDN]

I disapprove, i dont see us on there lol.

[u/KorporalKarnage → u/LumberjackCDN]

Heh. Yeah, retailers were out of the scope for my map project. But if you sell online I do have a used/consignment section on the GunsOnline.ca site!

Edit: lol Nevermind! You're already on there! Just needed a better pagination. My bots run different on each retailer, they all scrape different.

[u/KorporalKarnage → u/KorporalKarnage]

One more thing to add, my scraper caps out at 500 items/day. It may take more time to add the rest of your catalog (I think my log shows 5876 items in store).

This is how I programmed my scrapers, do a "light" touch every night so they don't get blocked (or my IP!).

[u/_Armpit → u/KorporalKarnage]

If it's 500 a day (per retailer or globally), you're going to be waiting weeks or months to index everything for some stores.

[u/KorporalKarnage → u/_Armpit]

I'm testing a few different proxy sites atm. Plus I'm making a method to run a scrape remotely from my mobile whenever I hit an open wifi connection (new IP). Basically a poor-man's proxy lol. I drive all over and have been saving wifi hotspots hehe.

Hat's off to you though! This isn't easy, and is still a work in progress.

[u/KorporalKarnage → u/_Armpit]

Proxy scraper now online. Should show over 13k items. Changed my scraper script to fall back on softer methods if kicked. Still working on more proxies...

[u/_Armpit → u/KorporalKarnage]

With the exception of Shopify and Reliable, most retailers' host don't care if you are making a lot of requests from a single IP address/identity.

I crawl most of my retailers from a single US datacenter IP and it's fine, you just have to watch those select few that display pricing in USD if you do that.

I end up crawling roughly 130k products every single day and it takes 5 hours to finish. I can go faster, but that's not nice bot behaviour for retailers.

[u/KorporalKarnage → u/_Armpit]

I've been having a blast trying different ways getting around all these anti bot/scraper blocks these sites put up. I had no idea there were so many ways they put up roadblocks but given time, they all fall down.

So my setup as of now is dedicated RPi running cron jobs for all the different tasks (and there are many). I also abandoned the porto-proxy app, although it did work, it just isn't needed with the setup as of today.

Basically it goes PC (if on)->Rpi->Proxy server 1->if problems->fallback proxy server 2-> Rpi->deploy

I upped my limits to 5k per for now and let er run....

[u/KorporalKarnage → u/KorporalKarnage]

Oh yeah, BassPro is NOT worth the effort! Holy....

[u/LumberjackCDN → u/KorporalKarnage]

Noticed lots of out of stock stuff being listed as in stock, at least from the one store.

[u/KorporalKarnage → u/LumberjackCDN]

Found the bug! Thanks for the heads up! BigCommerce and WC/Shopify needed tightening.

If you find it still there (I dunno where you found it) or find any more lemme know. It might take a while to re-scrape all BigCommerce stores...

[u/LumberjackCDN → u/KorporalKarnage]

Will do!

[u/LumberjackCDN → u/KorporalKarnage]

Excellent

[u/Technoaddict]

https://giphy.com/gifs/aYhxFdZu9ZQJzNkPqJ

[u/Anonymous-Old-Fart]

Well done!

[u/Equivalent-Shoe-1652]

This is great. Hopefully people won't be too stingy with their shooting spots.

[u/KorporalKarnage → u/Equivalent-Shoe-1652]

I know, right?

Once I've settled into a routine with the rest of the big stuff I'm gonna start posting the spots I know. I dgaf if "everyone" knows "my" spots, just don't make a friggin mess!!

[u/Equivalent-Shoe-1652 → u/KorporalKarnage]

Where you located?

[u/KorporalKarnage → u/Equivalent-Shoe-1652]

BC. Lower Mainland.

But know a lot of spots all over.

[u/Equivalent-Shoe-1652 → u/KorporalKarnage]

Ohh shit, im in Vancouver. If you feel like hitting some crownland or passing on some spots. Im keen.

[u/onenifty → u/Equivalent-Shoe-1652]

Yo, me too!

[u/One-Garage1260]

Filters don't appear to show on mobile for gunsonline

[u/KorporalKarnage → u/One-Garage1260]

Thanks for the bug! Fixed!

Let me know if the newer mobile layout for filters works for you. You will need to refresh the page.

(That bug was deeper than I thought!)

[u/One-Garage1260 → u/KorporalKarnage]

Working now. Bit of a weird issue where I'm seeing a Benelli Super Nova listed as $100 on the site, but when you click through to the retailer it shows $750 down from $850. Makes me think it's displaying the discount amount instead of the price? I checked two other listings and the issue didn't occur. I have screenshots I can send if you want to DM me.

[u/KorporalKarnage → u/One-Garage1260]

Fixed.

It was an issue with how the retailer posted the discount. The bug was found in only a few discount entries. Changed it so discounts are showing new price.

[u/RoundRefuse1740]

Wolf’s den only has an archery range

[u/KorporalKarnage → u/RoundRefuse1740]

Removed it. Thanks!

[u/squishydonut16]

The map is a really nice feature, especially if we get more user submitted crown spots

[u/Agent_1812]

There was no way to move a location, so I added a duplicate location for Halton Rod & Gun Club where it actually is (not in town).

[u/RoundRefuse1740]

The gun club that is in penetanguishene is wrong, that club is in Hamilton

[u/KorporalKarnage → u/RoundRefuse1740]

Fixed. Thanks!