ServerPortfolio  2.0
Python parsers and server
 All Classes Namespaces Files Functions Variables Properties Pages
Classes | Variables
serverportfolio.AutoParser Namespace Reference

Module for updating the dictionary of stocks using independent threads every TIMEPARSER sec,
and parsing data on demand from the server. More...

Classes

class  thread_parser
 Independent thread parsing the web pages, consuming data from the queue. More...
 
class  Thread_Handle_Queue
 Thread to handle the queueing system and create the thread_parsers. More...
 

Variables

int TIMEPARSER = 30
 Wait X seconds between parsing again. More...
 
int NB_THREADS_PARSER = 3
 Number of threads waiting for data in the queue. More...
 
tuple logger = logging.getLogger('SP')
 
tuple main_logger = logging.getLogger("SP.main")
 
tuple dict_stocks = DictionaryStocks( load_all_stocks=True )
 
tuple tqh = Thread_Handle_Queue()
 

Detailed Description

Module for updating the dictionary of stocks using independent threads every TIMEPARSER sec,
and parsing data on demand from the server.

Module loops over all stocks and record value in a 'CSV' style (in /data/historical/*INST.dcsv). To extend to different action, may use threads for other parsing actions (FORCE...)

Implement a queueing system of stocks which are regularly 'parsed' by independent threads:

The parser threads waits for data in the queue implements a simple lock if Server wait for an answer

The queue is run in a separate thread ( no daemon ), which creates 2 threads ( "daemon-ised" ) for parsing this way, main thread is still free

Add list_error_stock, easy to deal, may check when Dictionary is reloaded for instance may try to define temporary problem ( internet connection / slow ) and error in key with error value...

Rules for the states States for stock CLOSED(inactive)->WAIT_OPEN(active,but not write)->OPEN(active and write)->CLOSED(inactive)

LastChanged

Id:
AutoParser.py 9 2015-04-02 23:27:25Z michael

Variable Documentation

tuple serverportfolio.AutoParser.dict_stocks = DictionaryStocks( load_all_stocks=True )

Definition at line 503 of file AutoParser.py.

tuple serverportfolio.AutoParser.logger = logging.getLogger('SP')

Definition at line 499 of file AutoParser.py.

tuple serverportfolio.AutoParser.main_logger = logging.getLogger("SP.main")

Definition at line 500 of file AutoParser.py.

int serverportfolio.AutoParser.NB_THREADS_PARSER = 3

Number of threads waiting for data in the queue.

It is fixed at the initialisation.

Definition at line 40 of file AutoParser.py.

int serverportfolio.AutoParser.TIMEPARSER = 30

Wait X seconds between parsing again.

Definition at line 38 of file AutoParser.py.

tuple serverportfolio.AutoParser.tqh = Thread_Handle_Queue()

Definition at line 513 of file AutoParser.py.