Oddly Posted June 6, 2018 Posted June 6, 2018 I'm scraping http://www.mtgtop8.com for data on magic the gathering decks. I will be using the data in a genetic algorithm to generate a deck for MTG based off pro statistics. The data scraping process takes a bit of time, so i'm looking for other machines to run this scrape script on so i can get data faster, anyone interested in helping me run a script for a few hours? You must have python3, and i will send you the setup process and script via a pm.
Oddly Posted June 6, 2018 Author Posted June 6, 2018 28 minutes ago, Damian666 said: give me moneys, then ill do it xD  i feel like i have enough computers and friends that i won't need to pay for it. Otherwise i'd just rent a couple servers for 2 days
Damian666 Posted June 6, 2018 Posted June 6, 2018 no offence bro, but then why ask here? Â just curious
Oddly Posted June 6, 2018 Author Posted June 6, 2018 8 minutes ago, Damian666 said: no offence bro, but then why ask here?  just curious cuz the more systems i can get the faster i can write the algorithms i need. right now it will take me a week to scrape all the data i need. I need to process 30,000 web pages, and its about 1 web page every 4-6 seconds.
Damian666 Posted June 6, 2018 Posted June 6, 2018 ok, but let me quote you: 9 minutes ago, Crest said: i feel like i have enough computers and friends that i won't need to pay for it. Otherwise i'd just rent a couple servers for 2 days  im confused
Oddly Posted June 6, 2018 Author Posted June 6, 2018 Just now, Damian666 said: ok, but let me quote you:  im confused I can do the entire process without paying for it on my own i i wanted to, the more systems i have the faster the collection process takes. I'm asking for people who have an extra system they're not using that they will let me run code on for a couple hours.
Damian666 Posted June 6, 2018 Posted June 6, 2018 whatever, I did not really mean the pay part, but as far as I see, your asking, while saying you don't need it. shrug  guess I just don't get it xD  carry on.
Recommended Posts
Create an account or sign in to comment
You need to be a member in order to leave a comment
Create an account
Sign up for a new account in our community. It's easy!
Register a new accountSign in
Already have an account? Sign in here.
Sign In Now