Boot the crawler

We will start our crawler by defining and calling the boot() method. The first crawler we start will create the project directory and the two data files. The code that needs to be added to the spider.py file, under the Spider class:

    @staticmethod  # defines the method as static
    def boot():
        create_project_dir(Spider.project_name)  # creates the project folder using the name provided by the user
        create_data_files(Spider.project_name,
                          Spider.base_url)  # creates data files and add the homepage to the queue file
        Spider.queue = file_to_set(
            Spider.queue_file)  # converts the links from the queue file to a set for faster operation
        Spider.crawled = file_to_set(Spider.crawled_file)  # converts the links from the crawled file to a set