Today is a great day to release some projects to the wild!
- XPath - use XPath to find different fields
- Pagination - if you'd like to grab a list of things that are paginated, that's also possible
- Custom functions - if you'd like to do something special with the fields before saving them to DB
- Leftover event related functionality - originally AgriCatch was designed for events, because of that there's some leftover functionality in that matter:
- days_on_page - you can give the timespan within the events of a single page are included (for example if it's a weekly page - 7)
- start_day & num_of_days - you can give the importer a default starting day, it will then attempt to replace the url with a timestamp (according ot a format mentioned in the website template). Example:
Would then try to move forward days_on_page days until it reached the limit (num_of_days).
- HTML & XML support
Most of the useful documentation for building importers is found in the Repository under agricatch/website.py
To import you simply run the command:
python manage.py doimport website_name --days=7
website_name refers to the name of the website in lowercase!
More info to come..