aboutsummaryrefslogtreecommitdiff
path: root/crawler (unfollow)
Commit message (Collapse)AuthorFilesLines
2019-02-06Add ability to update url who are one week old and content modifiedneodarz4-2/+101
2019-02-06Fix logging infoneodarz2-4/+2
2019-02-06Add date when page are crawledneodarz2-2/+7
2019-02-03Add ability to update a page in db if updatedneodarz4-6/+10
2019-01-19Add khanindex nevrax indexationneodarz7-14/+83
2019-01-17Be more specific on index source databaseneodarz1-5/+5
2019-01-13Remove bullshitneodarz1-7/+0
2019-01-13Save all crawled datas in databaseneodarz2-1/+16