settings.py\neodarznet\crawler - assos/khaganat/khanindexer - Simple search engine of khaganat project - Mirror of https://git.khaganat.net/neodarz/khanindexer

about summary refs log tree commit diff

path: root/crawler/neodarznet/settings.py

blob: 8d65b095273d6d812edab0f8efc3c3d650167d01 (plain)

# -*- coding: utf-8 -*-

BOT_NAME = 'neodarznet'

SPIDER_MODULES = ['crawler.neodarznet.spiders']
NEWSPIDER_MODULE = 'crawler.neodarznet.spiders'

ROBOTSTXT_OBEY = True

DEPTH_LIMIT = 0