Now we can actually create our model and some users. We don’t want to store our users passwords in plain text in our database and so we will
So we have our controller, but we don’t have any users or any way to check a user. We also need to configure CodeIgniter to use sessions so that you don’t have to login to every single page you visit. To do this we will need to edit the config files in the application/config directory.
So now that we have a basic understanding of what the different parts of CodeIgniter are, we can get down to writing our first controller. The login controller. As we don’t want just anyone being able to access our site.
CodeIgniter is a powerful PHP framework with a very small footprint, built for developers who need a simple and elegant toolkit to create full-featured web applications.
Now that we can download our test page we need to parse the HTML so that it can be read easily by our program. This tutorial will give you a basic overview of the python module Beautiful Soup.
Now that we have the beginnings of a web scraper we need to test that it works. The following site is a test site for web scraping.
Our download function currently doesn’t do much in the way of retying downloads. In this next part we will add in some code to make our function try and download the page 3 times if it fails.