ServerPortfolio
2.0
Python parsers and server
|
Store data and functions related to one stock. More...
Public Member Functions | |
def | __init__ |
Constructor. More... | |
def | is_valid |
Return if the Stock is valid. More... | |
def | add_dict_stock |
Function to update Stock with the data extracted by the Parsers. More... | |
def | pre_process |
Pre-processing only for HistPrice, may extend to all 'csv' later. More... | |
def | post_process |
Post process the update: check, validate and save the data after one (unique) action is performed. More... | |
def | time_to_open |
Compute in how long time the stock will open, to test for setting the state 'WAIT_OPEN' in AutoParsers. More... | |
def | test_ML_done |
Check date and time for ML(Marche libre) specifically. More... | |
def | print_stock |
Create a one line string with the values stored in a dictionary. More... | |
def | read_xml |
ReadXML data from file into internal xml_dict. More... | |
def | write_xml |
Check the validity of the new retrieved data (in _dict_stock), store update in _dict_interactive. More... | |
Getter/Setter for internal dictionaries, provide initialisation on request | |
def | get_action |
Return the action template part from _dict_stock or one of its data value. More... | |
def | set_action |
Fill _dict_stock['action'] with new_data. More... | |
def | get_config |
return the 'config' dictionary or one of its key from _dict_stock More... | |
def | saved_as |
query the entry 'config' : 'save' of an action If specifc is indicated return a boolean value More... | |
def | get_dict_xml |
Return the dictionary of data for XML input/output. More... | |
def | set_dict_xml |
Similar to set_action with extended dictionary. More... | |
def | get_error |
Return a formatted error. More... | |
Use C++ wrapPyStock.so library to read/write CSV data to file | |
All functions check for the global variable GlobalDicts.TEST_MODE to write output in tmp directory | |
def | save_inst_value |
Save instantaneous values into a file. More... | |
def | last_CSV |
Get the date of the last data from the file saved on disk. More... | |
def | save_hist_price |
Save historical price in CSV format in ROOT_data/hp/symbol.DAY.CSV Use C++ wrapPyStock.UpdateCSV function. More... | |
Static Public Member Functions | |
def | to_list |
Similar to Utils.to_list but deals with Stock or list of Stocks. More... | |
Public Attributes | |
valid_update | |
Validation is done by a ValidStockUpdate object. More... | |
Static Public Attributes | |
tuple | sub_tree_action = self._create_xml_action( key_action, value_action ) |
Private Member Functions | |
def | _add_instvalue_data |
Update InstValue data. More... | |
def | _add_histprice_data |
Update Historical Price data. More... | |
def | _fill_dict_xml |
Create an empty template in _dict_xml and fill with the data in the xml file. More... | |
def | _create_xml_action |
Helper function, generate xml sub-tree for one action. More... | |
Private Attributes | |
_logger | |
_symbol | |
Main key for all dictionaries. More... | |
_dict_stock | |
Set of templates to store results from parsers, include 'Static' data. More... | |
_dict_xml | |
Dictionary for storing data for XML input/output, extends _dict_parser templates. More... | |
_dict_interactive | |
Store old/new values before update which may be applied to XML. More... | |
Functions/properties to access some common data from _dict_parser (Static and InstValue) | |
state = property(get_state, set_state) | |
Define both setter and getter, can use stock.state = new_state or print stock.state. More... | |
def | symbol |
Getter for the symbol. More... | |
def | symbol |
Setter raises an exception, it cannot be changed. More... | |
def | get_state |
Getter function for the state. More... | |
def | set_state |
def | get_market |
Get the market on which the stock belongs. More... | |
def | get_last_modification |
Return the last modification of the Instantaneous value. More... | |
Store data and functions related to one stock.
Stock is a collection of dictionary templates copied from StockTemplates:
Provides functions to read/save data:
Functions for querying and updating data from the web are implemented in UpdateStocks
Stock could have derived class (asset, FX, indice...), only done InvalidStock
def serverportfolio.Stock.Stock.__init__ | ( | self, | |
symbol, | |||
dict_static = None |
|||
) |
Constructor.
Creates the stock and save the 'Static' data read in the configuration/definition file "dictstocks.txt".
Other (sub-)dictionaries will be created on demand: get_dict_parser(), get_dict_xml()
If needed, one Stock can be constructed this way:
But the use of DictionaryStocks is more convenient to store multiple Stock objects.
symbol | of the stock as defined in dictstocks.txt |
dict_static | 'Static' template containing the data read from dictstocks.txt |
|
private |
Update Historical Price data.
The field date (last date) must be updated correctly
Definition at line 473 of file Stock.py.
References serverportfolio.Stock.Stock.get_action().
Referenced by serverportfolio.Stock.Stock.add_dict_stock().
|
private |
Update InstValue data.
With server running, the state (OPEN/CLOSED) must be updated correctly
dict_new_data,templ_action |
Definition at line 433 of file Stock.py.
References serverportfolio.Stock.Stock.get_action(), serverportfolio.Stock.Stock.set_action(), serverportfolio.Stock.Stock.state, serverportfolio.Parsers.YahooCSV.YahooCSV.symbol, and serverportfolio.Stock.Stock.symbol().
Referenced by serverportfolio.Stock.Stock.add_dict_stock().
|
private |
Helper function, generate xml sub-tree for one action.
key_action | action name |
value_action | associated dictionary to the dictionary key key_action |
Definition at line 1098 of file Stock.py.
References serverportfolio.Parsers.YahooCSV.YahooCSV.symbol, and serverportfolio.Stock.Stock.symbol().
|
private |
Create an empty template in _dict_xml and fill with the data in the xml file.
element | etree.Element corresponding to an action |
Definition at line 858 of file Stock.py.
Referenced by serverportfolio.Stock.Stock.read_xml().
def serverportfolio.Stock.Stock.add_dict_stock | ( | self, | |
dict_new_data, | |||
source | |||
) |
Function to update Stock with the data extracted by the Parsers.
Update _dict_stock with the new_data from Parser._list_return_data
dict_new_data | dictionary ('template' part only) containing the data to insert, similar template format that Stock._dict_stock |
source | name of the parser used to retrieved the data, inserted in _dict_stock |
Definition at line 405 of file Stock.py.
References serverportfolio.Stock.Stock._add_histprice_data(), serverportfolio.Stock.Stock._add_instvalue_data(), serverportfolio.Stock.Stock.get_action(), and serverportfolio.Stock.Stock.set_action().
def serverportfolio.Stock.Stock.get_action | ( | self, | |
action = None , |
|||
key = None |
|||
) |
Return the action template part from _dict_stock or one of its data value.
If action is None, return the root dictionary _dict_stock
If key is None return _dict_stock[action]['template']
If _dict_stock[ action ] is not existing, create an 'action' entry with the default template and return a link
If the key is not exiting, it is created (the action as well if needed), set it up to None and return it
The purpose of this design is to provide lazy initialiation of the action / key, but some care must be taken when calling the function,
get functions can modify the internal dictionary
Example of usage:
action | optional string corresponding to a GlobalDicts.EAction name |
key | optional string to get the value of _templ_action['template'][key] |
Definition at line 182 of file Stock.py.
References serverportfolio.Stock.Stock._dict_stock, and serverportfolio.Stock.Stock.get_action().
Referenced by serverportfolio.Stock.Stock._add_histprice_data(), serverportfolio.Stock.Stock._add_instvalue_data(), serverportfolio.Stock.Stock.add_dict_stock(), serverportfolio.ValidStockUpdate.ValidStockUpdate.check_list_csv(), serverportfolio.ValidStockUpdate.ValidStockUpdate.check_new_xml_data(), serverportfolio.ValidStockUpdate.ValidStockUpdate.gen_part_of_list_csv(), serverportfolio.Stock.Stock.get_action(), serverportfolio.Stock.Stock.get_error(), serverportfolio.Stock.Stock.get_last_modification(), serverportfolio.Stock.Stock.get_market(), serverportfolio.Stock.Stock.get_state(), serverportfolio.Stock.Stock.last_CSV(), serverportfolio.Stock.Stock.print_stock(), serverportfolio.Stock.Stock.save_hist_price(), serverportfolio.Stock.Stock.save_inst_value(), serverportfolio.Stock.Stock.set_action(), and serverportfolio.Stock.Stock.test_ML_done().
def serverportfolio.Stock.Stock.get_config | ( | self, | |
action, | |||
key = None |
|||
) |
return the 'config' dictionary or one of its key from _dict_stock
Definition at line 254 of file Stock.py.
References serverportfolio.Stock.Stock._dict_stock.
Referenced by serverportfolio.Stock.Stock.saved_as().
def serverportfolio.Stock.Stock.get_dict_xml | ( | self, | |
action = None |
|||
) |
Return the dictionary of data for XML input/output.
Similar to get_action but templates are extended with sub-dictionary
Always return a valid dictionary, maybe empty
action | name or EAction |
PortfolioError | if invalid action |
Definition at line 309 of file Stock.py.
References serverportfolio.Stock.Stock._dict_xml.
Referenced by serverportfolio.Stock.Stock.read_xml(), serverportfolio.Stock.Stock.set_dict_xml(), and serverportfolio.Stock.Stock.write_xml().
def serverportfolio.Stock.Stock.get_error | ( | self, | |
action = None |
|||
) |
Return a formatted error.
To generalise to action as well ? Only convenient way to get them from C++ ?
Definition at line 335 of file Stock.py.
References serverportfolio.Stock.Stock.get_action().
def serverportfolio.Stock.Stock.get_last_modification | ( | self | ) |
Return the last modification of the Instantaneous value.
Definition at line 390 of file Stock.py.
References serverportfolio.Stock.Stock.get_action().
def serverportfolio.Stock.Stock.get_market | ( | self | ) |
Get the market on which the stock belongs.
e.g. US, FR...
Definition at line 386 of file Stock.py.
References serverportfolio.Stock.Stock.get_action().
Referenced by serverportfolio.Stock.Stock.time_to_open().
def serverportfolio.Stock.Stock.get_state | ( | self | ) |
Getter function for the state.
Possible state OPEN/CLOSED/WAIT_OPEN, default CLOSED.
Definition at line 371 of file Stock.py.
References serverportfolio.Stock.Stock.get_action().
def serverportfolio.Stock.Stock.is_valid | ( | self | ) |
Return if the Stock is valid.
InvalidStock implements the same function, but return False
def serverportfolio.Stock.Stock.last_CSV | ( | self, | |
action | |||
) |
Get the date of the last data from the file saved on disk.
Use C++ code with wrapStock.so, including TEST_MODE variable for unit-test
Tested for daily data only.
file searched in /home/michael/workspace_kepler/ROOT_application/data_test/historical/CAC40.DAY.csv (with TEST_MODE) Store date as datetime in tmeplate HistPrice
Definition at line 568 of file Stock.py.
References serverportfolio.Stock.Stock.get_action(), serverportfolio.Stock.Stock.set_action(), serverportfolio.Parsers.YahooCSV.YahooCSV.symbol, and serverportfolio.Stock.Stock.symbol().
Referenced by serverportfolio.Stock.Stock.pre_process().
def serverportfolio.Stock.Stock.post_process | ( | self, | |
e_action, | |||
option_post | |||
) |
Post process the update: check, validate and save the data after one (unique) action is performed.
Delegate this functionality to a ValidStockUpdate object
e_action | EAction executed by a parser |
option_post | post-process option coming from UpdateStocks/RunParser |
Definition at line 498 of file Stock.py.
References serverportfolio.Stock.Stock.valid_update.
def serverportfolio.Stock.Stock.pre_process | ( | self, | |
e_action | |||
) |
Pre-processing only for HistPrice, may extend to all 'csv' later.
The function is called by AbstractParser.store_stock_copy
e_action | EAction to process |
Definition at line 488 of file Stock.py.
References serverportfolio.Stock.Stock.last_CSV().
def serverportfolio.Stock.Stock.print_stock | ( | self, | |
action | |||
) |
Create a one line string with the values stored in a dictionary.
If the dictionary is not loaded, return a string message with the symbol and a message starting with 'ERROR'
Usually called by DictionaryStocks.make_print_stocks() or UpdateStocks
action | the EAction name of the data to retrieve |
Definition at line 707 of file Stock.py.
References serverportfolio.Stock.Stock._dict_stock, serverportfolio.Stock.Stock.get_action(), serverportfolio.Stock.Stock.state, serverportfolio.Parsers.YahooCSV.YahooCSV.symbol, and serverportfolio.Stock.Stock.symbol().
def serverportfolio.Stock.Stock.read_xml | ( | self | ) |
ReadXML data from file into internal xml_dict.
Definition at line 778 of file Stock.py.
References serverportfolio.Stock.Stock._dict_xml, serverportfolio.Stock.Stock._fill_dict_xml(), serverportfolio.Stock.Stock.get_dict_xml(), serverportfolio.Parsers.YahooCSV.YahooCSV.symbol, and serverportfolio.Stock.Stock.symbol().
def serverportfolio.Stock.Stock.save_hist_price | ( | self | ) |
Save historical price in CSV format in ROOT_data/hp/symbol.DAY.CSV Use C++ wrapPyStock.UpdateCSV function.
Function improved flexibility can reread/append, deal with partial overlaps.
Definition at line 607 of file Stock.py.
References serverportfolio.Stock.Stock._dict_interactive, serverportfolio.ValidStockUpdate.ValidStockUpdate.gen_part_of_list_csv(), serverportfolio.Stock.Stock.get_action(), serverportfolio.Parsers.YahooCSV.YahooCSV.symbol, and serverportfolio.Stock.Stock.symbol().
def serverportfolio.Stock.Stock.save_inst_value | ( | self | ) |
Save instantaneous values into a file.
Use C++ wrapPyStock.so module
C++ insures a correct format, correct header, correct directory (including TEST_MODE for debugging and unit-test)
Definition at line 529 of file Stock.py.
References serverportfolio.Stock.Stock.get_action(), serverportfolio.Parsers.YahooCSV.YahooCSV.symbol, and serverportfolio.Stock.Stock.symbol().
def serverportfolio.Stock.Stock.saved_as | ( | self, | |
action, | |||
specific = None |
|||
) |
query the entry 'config' : 'save' of an action If specifc is indicated return a boolean value
If specific is None return the list or False
Check first for the entry in _dict_stock, if not available query the default template through StTmpl.saved_as
action | |
specific | optional entry, valid values are 'csv' and 'xml' |
Definition at line 282 of file Stock.py.
References serverportfolio.Stock.Stock.get_config().
Referenced by serverportfolio.ValidStockUpdate.ValidStockUpdate.check_new_xml_data().
def serverportfolio.Stock.Stock.set_action | ( | self, | |
action, | |||
key, | |||
new_data | |||
) |
Fill _dict_stock['action'] with new_data.
If already existing overwrite the data, otherwise create the necessary action
action | name |
key | name of the data |
new_data | to insert into _dict_stock[action]['template'] or ['template']['key'] |
PortfolioError | if wrong action |
Definition at line 222 of file Stock.py.
References serverportfolio.Stock.Stock._dict_stock, and serverportfolio.Stock.Stock.get_action().
Referenced by serverportfolio.Stock.Stock._add_instvalue_data(), serverportfolio.Stock.Stock.add_dict_stock(), serverportfolio.Stock.Stock.last_CSV(), and serverportfolio.Stock.Stock.set_state().
def serverportfolio.Stock.Stock.set_dict_xml | ( | self, | |
action, | |||
new_dict_xml | |||
) |
Similar to set_action with extended dictionary.
Definition at line 327 of file Stock.py.
References serverportfolio.Stock.Stock._dict_xml, and serverportfolio.Stock.Stock.get_dict_xml().
def serverportfolio.Stock.Stock.set_state | ( | self, | |
state | |||
) |
Definition at line 375 of file Stock.py.
References serverportfolio.Stock.Stock.set_action().
def serverportfolio.Stock.Stock.symbol | ( | self | ) |
Getter for the symbol.
Definition at line 362 of file Stock.py.
References serverportfolio.Stock.Stock._symbol.
Referenced by serverportfolio.Stock.Stock._add_instvalue_data(), serverportfolio.Stock.Stock._create_xml_action(), serverportfolio.Stock.Stock.last_CSV(), serverportfolio.Stock.Stock.print_stock(), serverportfolio.Stock.InvalidStock.print_stock(), serverportfolio.Stock.Stock.read_xml(), serverportfolio.Stock.Stock.save_hist_price(), serverportfolio.Stock.Stock.save_inst_value(), serverportfolio.Stock.Stock.symbol(), and serverportfolio.Stock.Stock.write_xml().
def serverportfolio.Stock.Stock.symbol | ( | self, | |
value | |||
) |
Setter raises an exception, it cannot be changed.
Definition at line 367 of file Stock.py.
References serverportfolio.Stock.Stock._symbol, and serverportfolio.Stock.Stock.symbol().
def serverportfolio.Stock.Stock.test_ML_done | ( | self | ) |
Check date and time for ML(Marche libre) specifically.
Stcks from the "Marche Libre" have only a quotation by day (15H30 in Paris).
Need only the data-date and time info If is 'open' assume than work has been done, to check...
stock | : name of the stock |
Definition at line 687 of file Stock.py.
References serverportfolio.Stock.Stock.get_action().
def serverportfolio.Stock.Stock.time_to_open | ( | self | ) |
Compute in how long time the stock will open, to test for setting the state 'WAIT_OPEN' in AutoParsers.
Definition at line 658 of file Stock.py.
References serverportfolio.Stock.Stock.get_market().
|
static |
Similar to Utils.to_list but deals with Stock or list of Stocks.
one | stock symbol (String) or Stock object, or a list of them |
def serverportfolio.Stock.Stock.write_xml | ( | self | ) |
Check the validity of the new retrieved data (in _dict_stock), store update in _dict_interactive.
Executed before an automatic or interactive update of the XML data (_dict_xml)
Updated data are stored in selff.dict_interactive
opt_interactive | optional ask validation to user def check_new_xml_data(self, opt_interactive = False): self._logger.debug("opt_interactive %s", opt_interactive) |
dict_interactive = {}
self._logger.debug("_dict_stock: %s", self._dict_stock)
#if self.get_dict_parser()['global_error']:
for key,value in self.get_action().iteritems() : self._logger.debug("key/value %s %s" % (key, value))
#if (key == 'Static') | (key == 'InstValue') :
#if not self.get_dict_xml( key )['config']['save_xml']:
if self.saved_as( key, 'xml' ):
dict_action_inter = self._valid_new_value_xml( key, opt_interactive )
if any(dict_action_inter): dict_interactive[ key ] = dict_action_inter
#else pass Final merge (after user/auto validation) and write XML file. This step includes the merging of the data in self.dist_interactive,
and (re-)write the file by a call to write_xml, include TEST_MODE for unit-test def save_new_xml_data(self): self._logger.debug("Entry save_new_data_to_xml")
for (key_action, value_action) in self._dict_interactive.iteritems(): self._logger.debug("key_action/value_action %s / %s" % (key_action, value_action))
dict_xml = self.get_dict_xml( key_action )
for (key_data, value_data) in value_action.iteritems(): self._logger.debug("\nkey_data / value_data %s / %s" % (key_data, value_data))
if value_data['repl_add'] > 0: self._logger.info("update xml value / key_data : %s / %s / %s " % \ (self.symbol, key_data, value_data['new_value']) )
# value can overwite the default, use the last one ...? #b_append_data = bool( dict_xml['template'][key_data][-1]['attrib']['append'] ) # Not working if new created file from template #if dict_xml['template'][key_data][-1]['attrib']['append'] == "True": if dict_xml['template'][key_data][-1]['attrib']['append'] == True: #b_append_data = True print "append data" new_tmpl_value = StTmpl.get_template_one_value_xml() new_tmpl_value['value'] = value_data['new_value'] new_tmpl_value['attrib']['date'] = value_data['date'] new_tmpl_value['attrib']['source'] = value_data['source'] new_tmpl_value['attrib']['append'] = 'True' # append to the list dict_xml['template'][ key_data ].append( new_tmpl_value ) # replace the last value, do not modify the previous entries else: dict_xml['template'][ key_data ][-1]['value'] = value_data['new_value'] # format timestamp for C++, transform on the fly ? date added automatically dict_xml['template'][ key_data ][-1]['attrib']['date'] = value_data['date'] dict_xml['template'][ key_data ][-1]['attrib']['source'] = value_data['source'] # delete entry ? just pass #else: self._logger.debug("dict_xml before wirte_xml:\n%s " % Utils.pretty_dict(self.get_dict_xml()) ) self._logger.debug("dict_xml before wirte_xml:\n%s " % (self.get_dict_xml()) ) self.write_xml()
Write the data stored in _dict_xml into a XML file. At this point, write only the data with a valid 'source', avoid to write the default template.
No parameter action, want to keep all previous and valid entries
Definition at line 1027 of file Stock.py.
References serverportfolio.Stock.Stock.get_dict_xml(), serverportfolio.Parsers.YahooCSV.YahooCSV.symbol, and serverportfolio.Stock.Stock.symbol().
|
private |
Store old/new values before update which may be applied to XML.
In case of interactive mode, it can be validated by the user could be in separate class Validation() only intermediate data
Definition at line 97 of file Stock.py.
Referenced by serverportfolio.ValidStockUpdate.ValidStockUpdate.check_new_data(), serverportfolio.ValidStockUpdate.ValidStockUpdate.post_process_one_action(), serverportfolio.Stock.Stock.save_hist_price(), serverportfolio.ValidStockUpdate.ValidStockUpdate.save_new_data(), and serverportfolio.ValidStockUpdate.ValidStockUpdate.valid_interactive().
|
private |
Set of templates to store results from parsers, include 'Static' data.
Definition at line 83 of file Stock.py.
Referenced by serverportfolio.ValidStockUpdate.ValidStockUpdate.check_new_xml_data(), serverportfolio.Stock.Stock.get_action(), serverportfolio.Stock.Stock.get_config(), serverportfolio.Stock.Stock.print_stock(), and serverportfolio.Stock.Stock.set_action().
|
private |
Dictionary for storing data for XML input/output, extends _dict_parser templates.
Initialised in read_xml()
Definition at line 91 of file Stock.py.
Referenced by serverportfolio.Stock.Stock.get_dict_xml(), serverportfolio.Stock.Stock.read_xml(), and serverportfolio.Stock.Stock.set_dict_xml().
|
private |
Main key for all dictionaries.
Definition at line 80 of file Stock.py.
Referenced by serverportfolio.Stock.Stock.symbol().
|
static |
serverportfolio.Stock.Stock.valid_update |
Validation is done by a ValidStockUpdate object.
Definition at line 99 of file Stock.py.
Referenced by serverportfolio.Stock.Stock.post_process().
Define both setter and getter, can use stock.state = new_state or print stock.state.
Definition at line 383 of file Stock.py.
Referenced by serverportfolio.Stock.Stock._add_instvalue_data(), and serverportfolio.Stock.Stock.print_stock().