Operators for reading and writing data

The class DataStock and derived (SimpleData, MultiData and DataCSV) overload standard friends operators (<< and >>) to facilitate coding.

To allow the use of pointers and references to the base class, member functions are also implemented and extended with the use of strings (convenient for unit-tests).
A consistent way to check the parsing of data is also available.
Most of the operators can combine 2 or more DataStocks as the standard operators >> and <<.

Reading values from a stream or string

A typical line we wish to read from a file contains the values and volume for a stock at a given date

# date               open     high     low      close    volume
2013-01-01 00:00:00, 3600.00, 3610.00, 3590.00, 3605.00, 1000000.00 

Attention

The format of input requires a fixed format of 19 characters for the date, followed by a comma (no space between).
The rest of the values must only be comma separated (with space or not).
The maximum size of the line is fixed to 256 in BDS.

The values should be stored in a candle

DataCSV data_csv; 

The volume in a SimpleData

SimpleData data_vol; 

Given than a stringstream, ss, containing the text line above, this can be done with the friend operator:

ss >> data_csv >> data_vol;

Or using member functions:

data_vol << ( data_csv << ss );

data_csv and data_vol will contain the correct date and values.

Note

The parentheses "()" are mandatory in this case to insure the correct order of execution.
The order of the object is also inverted (this may change later).
The stream must be in input and output mode, because it is internally modified .

The advantage of the member functions is to be able to use pointers to the base class.

DataStock *p_csv = &data_csv;
DataStock *p_vol = &data_vol;
*p_vol << ( *p_csv << ss );

Whereas friend functions need an explicit cast

ss >> static_cast<SDataCSV &>(*p_csv) >> static_cast<SimpleData &>(*p_vol); 

If the command was executed successfully:

• ss.eof() = true, means than all data from the stream have been used.
• ss.fail() = false, means than enought data was present to fill correctly the DataStock(s).

It can be tested with:

( is.eof() == true && is.fail() == false ) == true

Attention

before reuse of the stringstream do not forget to clear the error flags:

ss.clear(); 

The member functions have also been overloaded to accept a string as input:

string str = ss.str()
data_vol << ( data_csv << str );

In this case the input string should be empty after a correct execution

str.empty() == true;

Note

All operators call indirectly the friend functions.
This function has been optimised in avoiding intermediate string or stringstream (copy or creation).
It uses only C-array buffer and a boost::interprocess::bufferstream is used as iostream.
The performance is increased, but not so much (see and test yourself src_cpp/test/main_performance_readfile.cpp),
but additional buffer size must be used. They are declared in the namespace BDS.

Reading from a file

This code shows the use of boost::interprocess::bufferstream which is slightly more efficient.
But a similar code can be used with std::getline() and a string or stringstream.

 // declare some buffers
 char buffer[BDS::BUFFER_STREAM];
 memset ( buffer, 0, BDS::BUFFER_STREAM );
 //can set with buffer here, or wait
 boost::interprocess::bufferstream mybufstream;

 while ( file.getline( buffer, BDS::BUFFER_STREAM ) ) {

    // reset error flags
    mybufstream.clear();
    // copy line to b, could use strlen(buffer)
    mybufstream.buffer( buffer, BDS::BUFFER_STREAM );
    mybufstream >> data_csv >> sdata;
    // check stream state is correct
    BOOST_CHECK_EQUAL ( (mybufstream.eof() == true && mybufstream.fail() == false), true );

    // insert datastocks to the list
    lcandle2.AddData( data_csv );
    lvol2.AddData( sdata );
}  

Writing data to a stream or string

Similarly to output formatted date and values, different overloads can be used:

stringstream ss_out;
string str;
ss_out << data_csv;
data_csv >> ss_out2;
data_csv >> str; 

Members function can use a reference or base pointer:

ss_out << static_cast<SimpleData &>(*pdata);
*p_csv >> ss_out2;
*p_csv >> str_out;

A single DataStock can be printed to the console:

std::cout << data_csv << std::endl;
std::cout << static_cast<DataCSV &>(*p_csv) << std::endl;

Most operators can combine DataStocks, the output will be correct for streams, but not for std::cout :

ss_out << data_csv << data_vol;
data_vol >> ( data_csv >> ss_out );
data_vol >> (data_csv >> str_out);

Attention

std::cout will not give the expected result !

std::cout << data_csv << data_vol;

will print:

2013-01-01 00:00:00, 3600.00, 3610.00, 3590.00, 3605.002013-01-01 00:00:00 1000000.00 

There are no tests than output was correct. Certainly less critical than reading data...

Like for reading, writing to file need an intermediate ostringstream which need to be reset

std::ofstream file;
std::ostringstream ss_out;

//Write output into the stringstream
*pdata2 >> ( *pdata >> ss_out );
ss_out << std::endl;

//Write stringstream to file
file << ss_out.str();

//Need to reset the stringstream, clear or come back at the origin
ss_out.str("");
//Or if sure size will be the same
ss_out.seekp( 0, std::ios_base::beg);
//Now the stringstream can be filled again...

All operators are unit-tested in src_cpp/data_stock_readcsv.cpp

Link to all related functions