ServerPortfolio
2.0
Python parsers and server
|
Module for updating the dictionary of stocks using independent threads every TIMEPARSER sec,
and parsing data on demand from the server.
More...
Classes | |
class | thread_parser |
Independent thread parsing the web pages, consuming data from the queue. More... | |
class | Thread_Handle_Queue |
Thread to handle the queueing system and create the thread_parsers. More... | |
Variables | |
int | TIMEPARSER = 30 |
Wait X seconds between parsing again. More... | |
int | NB_THREADS_PARSER = 3 |
Number of threads waiting for data in the queue. More... | |
tuple | logger = logging.getLogger('SP') |
tuple | main_logger = logging.getLogger("SP.main") |
tuple | dict_stocks = DictionaryStocks( load_all_stocks=True ) |
tuple | tqh = Thread_Handle_Queue() |
Module for updating the dictionary of stocks using independent threads every TIMEPARSER sec,
and parsing data on demand from the server.
Module loops over all stocks and record value in a 'CSV' style (in /data/historical/*INST.dcsv). To extend to different action, may use threads for other parsing actions (FORCE...)
Implement a queueing system of stocks which are regularly 'parsed' by independent threads:
The parser threads waits for data in the queue implements a simple lock if Server wait for an answer
The queue is run in a separate thread ( no daemon ), which creates 2 threads ( "daemon-ised" ) for parsing this way, main thread is still free
Add list_error_stock, easy to deal, may check when Dictionary is reloaded for instance may try to define temporary problem ( internet connection / slow ) and error in key with error value...
Rules for the states States for stock CLOSED(inactive)->WAIT_OPEN(active,but not write)->OPEN(active and write)->CLOSED(inactive)
LastChanged
tuple serverportfolio.AutoParser.dict_stocks = DictionaryStocks( load_all_stocks=True ) |
Definition at line 503 of file AutoParser.py.
tuple serverportfolio.AutoParser.logger = logging.getLogger('SP') |
Definition at line 499 of file AutoParser.py.
tuple serverportfolio.AutoParser.main_logger = logging.getLogger("SP.main") |
Definition at line 500 of file AutoParser.py.
int serverportfolio.AutoParser.NB_THREADS_PARSER = 3 |
Number of threads waiting for data in the queue.
It is fixed at the initialisation.
Definition at line 40 of file AutoParser.py.
int serverportfolio.AutoParser.TIMEPARSER = 30 |
Wait X seconds between parsing again.
Definition at line 38 of file AutoParser.py.
tuple serverportfolio.AutoParser.tqh = Thread_Handle_Queue() |
Definition at line 513 of file AutoParser.py.